-- Logs begin at Wed 2022-09-28 06:39:05 UTC, end at Wed 2022-09-28 07:44:22 UTC. -- Sep 28 07:09:29.244921 dv-cinder-859447-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 28 07:09:29.254806 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 28 07:09:29 2022] *** Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: nodename: dv-cinder-859447-1-iscsi Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: machine: x86_64 Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: clock source: unix Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: pcre jit disabled Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: detected number of CPU cores: 4 Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: current working directory: / Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: detected binary path: /usr/bin/uwsgi-core Sep 28 07:09:29.257498 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: your processes number limit is 47680 Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: your memory page size is 4096 bytes Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: detected max file descriptor number: 2048 Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: lock engine: pthread robust mutexes Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: thunder lock: enabled Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 28 07:09:29.258207 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 28 07:09:29.296780 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: Python main interpreter initialized at 0x55d42570d9c0 Sep 28 07:09:29.296780 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: python threads support enabled Sep 28 07:09:29.296780 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: your server socket listen backlog is limited to 100 connections Sep 28 07:09:29.296780 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: your mercy for graceful operations on workers is 90 seconds Sep 28 07:09:29.297106 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: mapped 403077 bytes (393 KB) for 2 cores Sep 28 07:09:29.297192 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: *** Operational MODE: preforking *** Sep 28 07:09:29.297463 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: *** uWSGI is running in multiple interpreter mode *** Sep 28 07:09:29.297463 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: spawned uWSGI master process (pid: 81453) Sep 28 07:09:29.297683 dv-cinder-859447-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 28 07:09:29.299124 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: spawned uWSGI worker 1 (pid: 81463, cores: 1) Sep 28 07:09:29.299124 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: spawned uWSGI worker 2 (pid: 81464, cores: 1) Sep 28 07:09:29.299124 dv-cinder-859447-1-iscsi devstack@keystone.service[81453]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 28 07:09:33.558000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 07:09:33.558000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Sep 28 07:09:33.558777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 07:09:33.558777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Sep 28 07:09:33.558896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 07:09:33.558896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: from cryptography.utils import int_from_bytes Sep 28 07:09:33.558896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 07:09:33.558896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: from cryptography.utils import int_from_bytes Sep 28 07:09:33.608237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 07:09:33.608395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 07:09:33.608481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 07:09:33.608575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 07:09:33.608653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] command line args: [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 07:09:33.608729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] command line args: [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 07:09:33.608845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 07:09:33.608984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 07:09:33.609074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ================================================================================ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 07:09:33.609151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ================================================================================ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 07:09:33.609285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] admin_token = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_dir = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_dir = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.609950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_source = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_source = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] debug = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] debug = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.610936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_config_append = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.611907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_config_append = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_dir = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_dir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.612881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.613946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.614993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.615722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] publish_errors = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] publish_errors = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.616882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] standard_threads = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] standard_threads = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.617947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_journal = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_journal = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.618908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_json = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_json = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_stderr = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_stderr = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_syslog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_syslog = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 07:09:33.619835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.619951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.620869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.621926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.622957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.623905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.624883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.625025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.625153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.625266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.625388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.626947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.627970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.628888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.629987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.630950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.631971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.632898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.633947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.634978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.635996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.636878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.637932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.638941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.639952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.640857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.641005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.641146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.641787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.641911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.642903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.643969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.644883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.external = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.645961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.external = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.password = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.token = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.646894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.password = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.token = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.647907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.648933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.649988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.650898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.651913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.652913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.653993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.654903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.655898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.656926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.657895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.658898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.659933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.660970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.661911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.662934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.663985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.664889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.665937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.666958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.667977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.668879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.669935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.670960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.671984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.672107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.672221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.672824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.672937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.673947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.674975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.675998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.676876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.677918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.678891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.679927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.680961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.681802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.682931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.683981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.684889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.685984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.686897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.687919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.688064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.688669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.688773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.688868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.688981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.689948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.690947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.691933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.692913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.693891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 07:09:33.694525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.694891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.695132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.695370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 07:09:33.695539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 07:09:33.723390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.723617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.723840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.724233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.724468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.724696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.732128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.732992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.760882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.761132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.761676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.761911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.814904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.815038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.863103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.863331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.864058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:33.864294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.020855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.021006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.021092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.021223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.021308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.044698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.044929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.045920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.046048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.046131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.046259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.058000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.058245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.070809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.070991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.071936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.072061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.072142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.072273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.072485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 07:09:34.088694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.088898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 07:09:34.089117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.089257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.089339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 07:09:34.089454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 07:09:34.089655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.089738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.089853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 07:09:34.089973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.090899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.100543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.100823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.101963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.108336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.108934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.112054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.112336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.112527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.112610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.112815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.112896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.113949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.114832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.115025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.115296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.137693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.138006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.139147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.139426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.139508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.139744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.144500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.144614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.144789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.144899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.145094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.145190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.151415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.151501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.151705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.151787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.157891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.158043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.158187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.158338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.158471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.158628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.158751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.158916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.159887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.160023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.176019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.176198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.176287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.176472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.180148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.180380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.183649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.183914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.183996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.184210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.188186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.188496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.191341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.191667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.194558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.194856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.194955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.195984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.196064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.196287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.211713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.211998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.212088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.212265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.212349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.212527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.212628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.212848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.219886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.220239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.221445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.221821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.222975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.223258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.223338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.223614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.225921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.226204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.226429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.226511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.226715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.226795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.226997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.227137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.227288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.227679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.238969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.239260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.239342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.239544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.239625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.239825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.239907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.240850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.241114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.256426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.256742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.259343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.259682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.262225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.262580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.263732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.264084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.265223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.265506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.265586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.265865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.265944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.266320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.272468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.272749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.273964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.274249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.276821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.277935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.278116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.286159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.286450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.286549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.286733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.286816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.287034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.293582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.293868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.293960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 07:09:34.294151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.294975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.295870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.296909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.297019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.297199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.297279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.298057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 07:09:34.332094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d42570d9c0 pid: 81463 (default app) Sep 28 07:09:34.332641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d42570d9c0 pid: 81464 (default app) Sep 28 07:09:34.335323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300434e-9c51-4557-a451-3886fea97145 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:34.335605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300434e-9c51-4557-a451-3886fea97145 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:34.335871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300434e-9c51-4557-a451-3886fea97145 None None] PATH_INFO: `/v3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:34.339995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Sep 28 07:09:34 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:41.115482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:41.116003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:41.116107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:41.120292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:41.126113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:41.126693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:41.126789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:41.198636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: INFO dbcounter [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] Registered counter for database keystone Sep 28 07:09:41.217693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG oslo_db.sqlalchemy.engines [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81463) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 07:09:41.224578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writer thread running {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 07:09:41.414890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81463) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 07:09:41.415182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81463) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 07:09:41.473460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:41.475017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:41.599622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 28 07:09:42.039798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:42.108876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: INFO dbcounter [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] Registered counter for database keystone Sep 28 07:09:42.130327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG oslo_db.sqlalchemy.engines [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81464) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 07:09:42.136603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writer thread running {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 07:09:42.259708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b12944-097f-4a00-b126-3785adf22652 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:42.263993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:42.264621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:42.264895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:42.265260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:42.275382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:42.331968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 07:09:42.332271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806611f-dde3-4592-8c90-dd256a33e736 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:42.342888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 07:09:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:09:42.348902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:42.366482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a8c75-33d7-4b9f-969e-f28c39d721cb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:42.371170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:42.372286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:42.372567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:42.372838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] PATH_INFO: `/v3/projects/712d7cadf2c0478da279cd0f1a4dd259` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:42.374481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] RBAC: Authorizing `identity:get_project(project_id=712d7cadf2c0478da279cd0f1a4dd259)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:42.429767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 07:09:42.430114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d511-a45d-441b-9bab-0c9a308343b5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:42.435035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Sep 28 07:09:42 2022] GET /identity/v3/projects/712d7cadf2c0478da279cd0f1a4dd259 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:44.596772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c1d1-68c8-4a6c-bdcf-93a4cc884664 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:44.597333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c1d1-68c8-4a6c-bdcf-93a4cc884664 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:44.597333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c1d1-68c8-4a6c-bdcf-93a4cc884664 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:44.601463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:44.606956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:44.607345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:44.607496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:44.631794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:44.633334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:44.641418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 28 07:09:44.911407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84c1d1-68c8-4a6c-bdcf-93a4cc884664 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:44.923727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84c1d1-68c8-4a6c-bdcf-93a4cc884664 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:44.927991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:44.928620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:44.928918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:44.929219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:44.936994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:44.939046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61501302-0717-4ca8-9605-caac27bd4b22 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:44.947670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 07:09:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:09:44.952854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:44.962719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b5f73-a464-4dcc-b065-629c2012e2b7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:44.966947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:44.967562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:44.967837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:44.968103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:44.970008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:44.971817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf80f72-1f96-4223-9cde-f337cb729a42 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:44.973356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Sep 28 07:09:44 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:47.192311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf9d8-e685-4f0e-9f32-808451d312a9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.192852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf9d8-e685-4f0e-9f32-808451d312a9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.192852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf9d8-e685-4f0e-9f32-808451d312a9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.194198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:09:47.199312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.199751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.199966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.218138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:47.219918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:47.221556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dd18-786b-4d78-a2be-ce8f6d640484 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.221876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dd18-786b-4d78-a2be-ce8f6d640484 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.222154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dd18-786b-4d78-a2be-ce8f6d640484 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.223457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:47.228685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.228996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.229274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.271029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG passlib.handlers.bcrypt [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81464) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 07:09:47.271415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG passlib.handlers.bcrypt [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81464) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 07:09:47.305491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:47.320778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:47.322374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:47.421052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:09:47.571467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:47.596770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:47.674678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9decfcf-424f-4ad0-b65e-6d22b571f366 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:47.688593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:47.709873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:47.710507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.710785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.711056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.722059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:09:47.730810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:09:47.731714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:09:47.732526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:09:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:09:47.737785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:47.789842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f02c0-4a80-4e6e-b771-f6a72f85e5fd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:47.792053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:47.821066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:47.827146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:47.828545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.828885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.829187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.839523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8a49b-c9f8-4396-b54e-362df9bd89db None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:47.872849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:47.873486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:47.873766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:47.874043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:47.874710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:47.875649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:47.877113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:47.888574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:09:47.895515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:47.897247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:47.898756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:47.997428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:09:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:09:47.999782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b6440-dcaa-4b5f-a259-d708a0a9c271 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:48.003853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:48.006604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:48.037069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:48.037708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:48.037986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:48.038261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:48.039422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:48.040873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:48.042461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:09:47 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:48.048437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:48.064124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:48.106083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453338b-d693-4289-aca5-89da1568b834 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:48.142214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:48.143066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:48.143344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:48.143620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:48.152387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f43-91f4-4ca0-9a02-bf56385d2697 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:48.162260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:48.163729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:09:48.182862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:09:48.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:09:48.184528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:09:48.184528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:09:48.184528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:09:48.184528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:09:48.186268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:48.186900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:48.187182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:48.188122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:48.189294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:48.190502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:48.190752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:48.192303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:09:48.197839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:48.246518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:48.279160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:48.292765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b733c97-0510-43b6-9b38-419bb3c2da7c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:48.300162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f153ef-aaa5-4444-9492-b50d94cdef3c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:48.326420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:48.327021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:48.327301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:48.327573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:48.328243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:48.328544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:48.329961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1861be1-2400-49db-bb50-c807fe76ff57 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:48.333458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:48.334058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:48.334337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:48.334611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:48.335330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.domain.id=default, user.id=955dc67368494986a8ad78d63a857238) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:48.335639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:48.337093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdafb-a3d0-4f87-aab5-35b2d23a4de1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:48.352290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 07:09:48 2022] GET /identity/v3/role_assignments?user.id=955dc67368494986a8ad78d63a857238&scope.domain.id=default&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:09:48.352511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:50.934370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b61a0-f253-4317-a336-6b2f5883b911 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:50.934871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b61a0-f253-4317-a336-6b2f5883b911 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:50.934943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b61a0-f253-4317-a336-6b2f5883b911 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:50.936584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:50.941871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346716-0c8d-40af-abd0-1bb07b99b817 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:50.942300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346716-0c8d-40af-abd0-1bb07b99b817 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:50.942457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346716-0c8d-40af-abd0-1bb07b99b817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:50.970145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-db346716-0c8d-40af-abd0-1bb07b99b817 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:50.971270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-db346716-0c8d-40af-abd0-1bb07b99b817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:51.060898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:09:51.100426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0398c-0b85-4a6f-9c43-d1767080b3d2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.100742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0398c-0b85-4a6f-9c43-d1767080b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.100992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0398c-0b85-4a6f-9c43-d1767080b3d2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.102461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:51.107738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23e33-c5a5-4c9c-9c16-4bee4586decd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.108024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23e33-c5a5-4c9c-9c16-4bee4586decd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.108296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23e33-c5a5-4c9c-9c16-4bee4586decd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.133943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-25d23e33-c5a5-4c9c-9c16-4bee4586decd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:51.135118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-25d23e33-c5a5-4c9c-9c16-4bee4586decd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:51.199081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:51.230146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 12/25] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:51.415474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3942-0e8b-4414-a926-05c4e6a44208 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.415860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3942-0e8b-4414-a926-05c4e6a44208 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.416059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3942-0e8b-4414-a926-05c4e6a44208 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.417379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 14/26] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:51.423000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b76b9c-e0bd-43d6-8285-3936b964a935 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.423283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b76b9c-e0bd-43d6-8285-3936b964a935 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.423558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b76b9c-e0bd-43d6-8285-3936b964a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.449937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e5b76b9c-e0bd-43d6-8285-3936b964a935 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:51.451119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5b76b9c-e0bd-43d6-8285-3936b964a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:51.474615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14c30-7159-4d4c-b392-9c754979ad31 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.474895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14c30-7159-4d4c-b392-9c754979ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.475153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14c30-7159-4d4c-b392-9c754979ad31 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.476482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 15/27] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:09:51.480544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad14c30-7159-4d4c-b392-9c754979ad31 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:51.576425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 13/28] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:51.578557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.578844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.579112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.610041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:51.611239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:51.641087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad14c30-7159-4d4c-b392-9c754979ad31 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:51.707553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:51.708200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.708477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.708746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.726700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:09:51.731673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:09:51.732473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:51.733240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:09:51.735484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 16/29] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:09:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:09:51.736781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:51.738651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 14/30] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:51.739496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:51.750924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:51.803651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:51.904456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9437a3-d693-4157-a7a7-221cc763a82c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:51.949704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72187-9005-4aef-a428-35889cc7bc59 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:51.992308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:51.993537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:51.995028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:51.995446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:51.996450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:51.997990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:51.999530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.022230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.023077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.023522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.023941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.026344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 15/31] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:09:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:52.028918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.050088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.051609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:09:52.077345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:09:52.079126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:09:52.079917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:09:52.079917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:09:52.079917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:09:52.079917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 17/32] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:09:52.080134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.144541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.175259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.181096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ed5c1-96df-4626-9862-244865356e76 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.191979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85ea6b-c1a1-4523-b873-42ab21bd1fb2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.221699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.222474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.223052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.223335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.224291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.225713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.227436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.228067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.228349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.228623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.229838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.231347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.232936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:09:52 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:52.233710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.339424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594cba2-1a0a-4289-a458-763d5345a62d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.360596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 07:09:52 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:09:52.362229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.365008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.377528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.378428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.378871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.379284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.380138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.381738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92543bd-0191-4c6e-9ed1-30f1d21e2d2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.484461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e06-a68c-46ae-b18a-459d7904c057 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.509919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 19/35] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 07:09:52 2022] POST /identity/v3/roles => generated 225 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:09:52.511764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b50f2-75b6-4087-b432-f15b4549e7fa None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.512194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b50f2-75b6-4087-b432-f15b4549e7fa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.512598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b50f2-75b6-4087-b432-f15b4549e7fa None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.514134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 20/36] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:52.515106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b50f2-75b6-4087-b432-f15b4549e7fa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.542836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.543444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.543817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.544182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.545037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:52.545440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.548192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451c0cd-e05a-4065-a69a-0b8f7572095e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.577161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:52.579405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.579749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.580029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.607025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:52.608613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:52.632090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.709819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b50f2-75b6-4087-b432-f15b4549e7fa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.742177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 18/38] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:52.745428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.761493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.762133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.762431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.762682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.763959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.766153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.768069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 21/39] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:52.773650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:52.841889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.893578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab5d8b-2993-4c94-adac-a7656ee6f1d1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.901230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e934831-b048-4144-9cd9-fb18aaf3cdd5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:52.939606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.940543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.941054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.941526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.942631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.944260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.948924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:52.949756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.950287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.950287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] PATH_INFO: `/v3/domains/default/users/955dc67368494986a8ad78d63a857238/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.952421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=955dc67368494986a8ad78d63a857238, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:52.954221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442530b4-5011-4bfe-a1a0-14c7d265d8f5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:52.972189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 22/40] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Sep 28 07:09:52 2022] PUT /identity/v3/domains/default/users/955dc67368494986a8ad78d63a857238/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:09:52.973284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7a11-a643-4fa1-9516-32b10f10e08a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.973618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7a11-a643-4fa1-9516-32b10f10e08a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.973899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7a11-a643-4fa1-9516-32b10f10e08a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.975313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 23/41] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:52.980463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:52.982976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:52.983632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:52.993028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:52.993697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 19/42] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:09:52 2022] POST /identity/v3/projects => generated 301 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:09:53.038041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:53.127192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:53.129123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:53.173934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d2e7-d4a5-4a32-b42f-b714538578cf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:53.215313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:53.216043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:53.216332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:53.216607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:53.217355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:53.219591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:53.221143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 24/43] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:53.236738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 20/44] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 07:09:53 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:09:53.500501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:53.543523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:53.601699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadedd8-047e-4c05-9ebd-80cdd48d1db1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:53.635086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:53.635722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:53.636002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:53.636273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:53.637409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:53.638854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ac37b-90d8-4fb7-a49a-0ee8911d2dd6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:53.641225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 25/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:53.646791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:53.708108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:53.749203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816903-f3c2-4234-9b69-1c9247746f0d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:53.782580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:53.783179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:53.783455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:53.783724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:53.784382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:53.785880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96315c-8d7b-45c6-9471-72af11a512fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:53.811171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 21/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:09:53 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:09:55.153111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aa716-a0cf-4093-831e-4b923d7facb6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.153629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aa716-a0cf-4093-831e-4b923d7facb6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.153734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aa716-a0cf-4093-831e-4b923d7facb6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.154992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 26/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:55.160543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.160919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.161247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.179229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:55.180401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:55.245692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 22/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:55.523610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aa716-a0cf-4093-831e-4b923d7facb6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:55.625654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aa716-a0cf-4093-831e-4b923d7facb6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:55.635962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:55.659511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:55.660132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.660410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.660679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.670968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:09:55.676319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:09:55.677152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:09:55.677895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 27/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:09:55.683502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:55.730611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:55.788351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d63ca-696c-4b1e-a59a-68dc7d016d05 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:55.822322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:55.822937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.823220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.823494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.824159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:55.824463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:55.825910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a47c6-7bba-4b45-b661-e4afc180741a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:55.833947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:55.840234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:55.945929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e948d-71d0-44b3-94b6-198426f1c751 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:55.967091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3872-7c03-4225-86f1-4235fad353b9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.967380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3872-7c03-4225-86f1-4235fad353b9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.967660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3872-7c03-4225-86f1-4235fad353b9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.969567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 24/51] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:55.974598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.974875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.975147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.979753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:55.980386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:55.980668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:55.980966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:55.982089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:55.983544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:55.986141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 28/52] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:09:55 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:55.991829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:56.036818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:56.097847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:56.098967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:56.103861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe044c-c827-490f-b699-95f482e1a367 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:56.138816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:56.139456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:56.139736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:56.140516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:56.141677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:56.143123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:56.144689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 29/53] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:09:56.150561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:56.167922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 25/54] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:09:56.246858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e63f8-0863-4b1e-a68a-03a9b82a0c5b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:56.253728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:56.283700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:56.284298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:56.284574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:56.284841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:56.285594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.domain.id=default, user.id=955dc67368494986a8ad78d63a857238) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:09:56.285899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:56.287319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:56.292889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 30/55] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 07:09:56 2022] GET /identity/v3/role_assignments?user.id=955dc67368494986a8ad78d63a857238&scope.domain.id=default&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:09:56.431413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:56.465788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:56.534275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215de404-67ba-4987-9236-f4e19f3a2705 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:56.567163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:56.567789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:56.568069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:56.568343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:56.569459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:56.570882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca9a62-4647-4175-931a-3e4ccfb562e7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:56.572445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 26/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:56.578896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:56.687574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce0dd-baf1-4a0c-b91f-ff7bb7cf1e48 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:56.720751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:56.721396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:56.721675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:56.721946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:56.722593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:56.724032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162649-13a9-4cb1-85e6-5787dabfd3ba None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:56.751483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 31/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 07:09:56 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:09:58.875339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0117-23c9-4893-9397-2fae5d24f5a3 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:58.875873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0117-23c9-4893-9397-2fae5d24f5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:58.875873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0117-23c9-4893-9397-2fae5d24f5a3 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:58.877206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 27/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:09:58.882367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:58.882756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:58.882914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:58.901076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:09:58.902254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:09:58.918023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:58.968899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 32/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:09:59.237230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c0117-23c9-4893-9397-2fae5d24f5a3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:59.336102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c0117-23c9-4893-9397-2fae5d24f5a3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:59.337621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:09:59.370510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:59.371130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:59.371407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:59.371676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:59.372772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:59.374277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5a071-1fc0-4f67-a012-26898c09e915 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:59.376586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 28/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:09:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:09:59.382191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:09:59.477746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df5fb-3352-4973-86e1-c2f9ac95633a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:09:59.510963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:09:59.511495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:09:59.511773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:09:59.512047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:09:59.512686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:09:59.514174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c11add-f762-4cc5-8735-c6d930f36d7e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:09:59.539703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 33/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:09:59 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:02.791649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bc9a-6ab2-4f8c-9ded-456a323957f3 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:02.792596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bc9a-6ab2-4f8c-9ded-456a323957f3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:02.793105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bc9a-6ab2-4f8c-9ded-456a323957f3 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:02.794659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 29/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:02.800283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e2b8c-f6ca-42f5-9673-b59878aa85f9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:02.801158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e2b8c-f6ca-42f5-9673-b59878aa85f9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:02.801627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e2b8c-f6ca-42f5-9673-b59878aa85f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:02.829930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f85e2b8c-f6ca-42f5-9673-b59878aa85f9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:02.831455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f85e2b8c-f6ca-42f5-9673-b59878aa85f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:02.833909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:02.920148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 34/63] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:03.041583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecc05-97d5-40b7-9e2b-f754b377859b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.041897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecc05-97d5-40b7-9e2b-f754b377859b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.042130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecc05-97d5-40b7-9e2b-f754b377859b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.043728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 30/64] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:03.049914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7431b-1e19-4389-b3b8-0760ab5af5b8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.050199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7431b-1e19-4389-b3b8-0760ab5af5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.050472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7431b-1e19-4389-b3b8-0760ab5af5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.078669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3ad7431b-1e19-4389-b3b8-0760ab5af5b8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:03.079927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3ad7431b-1e19-4389-b3b8-0760ab5af5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:03.174891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 35/65] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:03.247661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824781-6c4a-4ad7-88a6-99c19970cd45 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.248280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824781-6c4a-4ad7-88a6-99c19970cd45 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.248744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824781-6c4a-4ad7-88a6-99c19970cd45 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.250457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 31/66] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:03.255835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe884-8a41-4d98-b392-1d5ee5ad4b42 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.256125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe884-8a41-4d98-b392-1d5ee5ad4b42 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.256405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe884-8a41-4d98-b392-1d5ee5ad4b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.269890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824781-6c4a-4ad7-88a6-99c19970cd45 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.280884 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-478fe884-8a41-4d98-b392-1d5ee5ad4b42 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:03.282423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-478fe884-8a41-4d98-b392-1d5ee5ad4b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:03.392267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:03.401810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 36/67] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:03.448297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d520fb-2e2d-44da-819c-71c0b0db40cb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.448797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d520fb-2e2d-44da-819c-71c0b0db40cb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.449265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d520fb-2e2d-44da-819c-71c0b0db40cb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.450732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 37/68] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:03.453193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824781-6c4a-4ad7-88a6-99c19970cd45 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:03.456264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.456747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.457247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.482999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:03.485293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:03.518997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:03.519990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.520482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.521526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.535169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2bbb8529-4725-4188-b6ec-9e3201df9a27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:03.541978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:03.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:03.543535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:03.545451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 32/69] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:03.549827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b915c-c318-46ce-b322-9aaf5efacd9d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.549827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b915c-c318-46ce-b322-9aaf5efacd9d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.549827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b915c-c318-46ce-b322-9aaf5efacd9d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.550084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 33/70] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:03.550963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b915c-c318-46ce-b322-9aaf5efacd9d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.571763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 38/71] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:03.572777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.595084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:03.676859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b915c-c318-46ce-b322-9aaf5efacd9d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:03.710243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6166-a92d-4545-b319-6f2fb2085ae3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:03.728069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:03.728703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.729010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.729288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.741757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:03.744458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d39836f2-0a1b-475c-a210-4c109c329ce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:03.747109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:03.747941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:03.748852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 34/72] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:03.749880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.750059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.750425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.756495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:03.757183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.757589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.757865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.758538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:03.758847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:03.760285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:03.774227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 39/73] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:03.774380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:03.774492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.774575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:03.845587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 35/74] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:03.845587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.867887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:03.880006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d5c0a-24d8-446c-9c0b-4910cd852a5d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:03.915411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:03.916069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:03.916353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:03.916621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:03.927416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:03.932936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:03.933947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:03.934683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 40/75] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:03.935292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:03.960201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbb56-4b47-4b2f-94e3-ed2b227c19ab None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:03.985551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.000598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.001318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.001599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.001867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.006005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:04.006321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.010669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.019808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 36/76] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.025175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.039199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247fd1c-71f8-4c46-9432-e205a60b6327 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.076870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.077870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.078352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.078794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.092629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.094822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.096572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 41/77] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:03 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.097513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.118455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.137692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15492c6-06dd-4df3-ba55-be74727a67f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.172992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.173617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.173896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.174167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.184367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:04.189930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:04.190765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:04.191752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 37/78] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:04.192698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.201208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ad34d-29f4-4ffd-b8d2-ec913356a387 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.211368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.237297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.237923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.238206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.238477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.239134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:04.239437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.240824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.249076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 42/79] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.249935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.296601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215f41-7abe-4abe-9fd5-b483bce10015 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.331353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.332031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.332328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.332595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.333988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.335424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.337190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 38/80] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.337767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.349338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99f08-0251-4014-9b26-232c1cfccbbe None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.350010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.383600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.384207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.384486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.384759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.424371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.435419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670234-f6f9-487e-b843-18236a3425b5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.454416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.455891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.468724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.469402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.469713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.469989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.471115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.472564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.474525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 39/81] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.475312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.553704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 43/82] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:04.554819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.554950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.580457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856db819-067f-4d1a-a981-adca5d8ff85a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.613538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.614141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.614423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.614697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.615363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:04.615667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.617056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.622333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.625309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 40/83] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.629878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.654225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653cef7-a3ae-4e32-a2cc-2df8b021d574 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.690126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.690694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.690982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.691259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.692407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.693916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.695500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 44/84] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:04.697822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.732746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c7c-bca1-4675-b257-858d22b93beb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.766532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.767140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.767419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.767693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.777801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.796713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ea10-f586-41ed-a226-18f9813394e6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.829646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:04.830245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:04.830521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:04.830793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:04.831440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.832868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.834256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:04.835718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:04.842515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:04.853802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 45/85] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:04 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:04.854830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:04.970388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb7fa-0052-4f04-b319-ad8be231b9e2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:04.975895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 41/86] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:04.975895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.013836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.014482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.015422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.015776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.017207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.019546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.021222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 46/87] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:05.022003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.026601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.095664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e91ff-5255-45e9-8571-7d02d49de7cf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.098119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.147962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7124-6239-414c-843d-867c9436c2f2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.150445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.151238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.151557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.151837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.153140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.154600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.156433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 42/88] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:04 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:05.161291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.188260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.189230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.189566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.189849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.271535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.273005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.286113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f3a5-9370-487d-9ef6-19d12d2e772c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.297931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.325934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.326790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.327653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.327947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.329502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.331517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.333855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 43/89] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:05.334479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.350860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.390235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 47/90] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:05.391556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.474425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004e3e-364c-474c-8c1b-cd9699f94246 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.511738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b9b9-b66c-4f53-bc36-bb7cb558fd9a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.516088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.516748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.517064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.517346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.518079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:05.518391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.519858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.528707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:05.529679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.586447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.600739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.601412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.601695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.601968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.653335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:05.686234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.688125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.739154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e100a4-c131-4581-b1f0-4334f3f5e586 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:05.809870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:05.810400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:05.810676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:05.810947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:05.812334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=955dc67368494986a8ad78d63a857238) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:05.812511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:05.814050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:05.828968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 45/92] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/role_assignments?user.id=955dc67368494986a8ad78d63a857238&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:05.829843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.871367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 48/93] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:05.877806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:05.990135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:06.026615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299510a7-6fdd-4279-b052-5d948138f2b0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:06.030068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:06.066961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea50c05-3fa0-40c0-aaa4-7bd6fa4fe9d2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:06.078961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:06.079867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:06.080292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:06.080730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] PATH_INFO: `/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:06.083945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] RBAC: Authorizing `identity:get_project(project_id=5a8c1b628c11460c88e0f5b699d2b66e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:06.086265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:06.088608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 46/94] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:06.095354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:06.158466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:06.159302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:06.159737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:06.160116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:06.165330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:06.167051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:06.169199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 49/95] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:05 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:06.175255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:06.236743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2188d-606d-454f-a7c1-d1da16d7811e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:06.286044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:06.286657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:06.286933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:06.287204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:06.287929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=5a8c1b628c11460c88e0f5b699d2b66e, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:06.288241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:06.289692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa88757-65c2-4efe-b803-b49b79c1166a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:06.300883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:06 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=5a8c1b628c11460c88e0f5b699d2b66e&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:06.346101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32464e2-5ae4-40ae-8637-ff6aac17cf4c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:06.391972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:06.398743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:06.399375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:06.399658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:06.399934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:06.400684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:06.401023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:06.402545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d720d-86ba-48de-8eaa-60db0aafaf63 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:06.410957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:06 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:07.945380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b3f6-f097-4d10-bbc9-776dda6b7eb3 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:07.945881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b3f6-f097-4d10-bbc9-776dda6b7eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:07.945953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b3f6-f097-4d10-bbc9-776dda6b7eb3 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:07.947244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:07.954300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b71d-9ada-4978-8bac-1614c61273d4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:07.954716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b71d-9ada-4978-8bac-1614c61273d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:07.954806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b71d-9ada-4978-8bac-1614c61273d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:07.986192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3f71b71d-9ada-4978-8bac-1614c61273d4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:07.987786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3f71b71d-9ada-4978-8bac-1614c61273d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:08.083316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:08.123748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b586-7f9d-4c36-8ae3-de6a1811f8ad None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.124586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b586-7f9d-4c36-8ae3-de6a1811f8ad None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.124836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b586-7f9d-4c36-8ae3-de6a1811f8ad None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.126608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 49/100] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:08.147097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e8eb-c83f-4c0a-8096-949b51245436 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.147403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e8eb-c83f-4c0a-8096-949b51245436 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.147656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e8eb-c83f-4c0a-8096-949b51245436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.174378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4ac9e8eb-c83f-4c0a-8096-949b51245436 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:08.175654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4ac9e8eb-c83f-4c0a-8096-949b51245436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:08.276801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 52/101] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:08.418376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860fe1-afda-46f8-95f1-1461efe3812d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.418711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860fe1-afda-46f8-95f1-1461efe3812d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.418934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860fe1-afda-46f8-95f1-1461efe3812d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.420534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 50/102] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:08.432562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.432854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.433166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.457949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:08.459160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:08.519785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:08.552686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 53/103] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:08.575197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860fe1-afda-46f8-95f1-1461efe3812d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:08.616074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:08.690842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860fe1-afda-46f8-95f1-1461efe3812d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:08.726100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:08.726749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.727032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.727310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.740608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:08.745638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:08.746465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:08.747622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 51/104] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:08.753172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:08.772621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:08.880981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:08.887794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26887c-e2bd-4877-b6c0-9cc2898a442c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:08.927975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:08.928753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:08.929067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:08.929347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:08.930720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:08.932252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:08.940582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 52/105] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:08.940726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e17ca9-6684-4a2b-b640-7b3dd149b871 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:08.942831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.002586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.003506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.003788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.004065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.004735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:09.005067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:09.006459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:09.024147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 54/106] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:09.031561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.121600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994c302-ec94-4f4c-a4c5-ee0e38b8e976 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:09.145423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:09.168336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.168939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.169873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.171255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.174467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:09.177395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:09.231394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 53/107] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:10:08 2022] POST /identity/v3/projects => generated 302 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:09.232255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.273862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ffc65-076c-4837-82de-826165f349b7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:09.279415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:09.368771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.369459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.369748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.370027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] PATH_INFO: `/v3/users/54e3afcc046c4c92837a47cd92b1aa2c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.405668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] RBAC: Authorizing `identity:get_user(user_id=54e3afcc046c4c92837a47cd92b1aa2c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:09.409116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fafd8-dee2-49fd-ac55-37645074e980 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:09.412879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 55/108] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:09 2022] GET /identity/v3/users/54e3afcc046c4c92837a47cd92b1aa2c => generated 289 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:09.422398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8b2a-6f23-46d6-b382-874b968d463d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.422702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8b2a-6f23-46d6-b382-874b968d463d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.422963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8b2a-6f23-46d6-b382-874b968d463d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.426783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 56/109] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:09.432931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.433798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.434083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.480542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:09.483694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35365b8-bd2a-436f-831a-1786ad43d37b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:09.486126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:09.549010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.552704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.553044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.553332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.574960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:09.581008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:09.581843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:09.582582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:09.583162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:09.583947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.617323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 57/111] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:09.625040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.680908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:09.720830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:09.770066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2c210-3a5d-40a2-b53f-78a7c67b8a0a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:09.838954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.839932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.840406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.840868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.856321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515353f-bcab-4ab2-a6d7-44df15bac87b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:09.923781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:09.924852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.925350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.925813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.926675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:09.927188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:09.931437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d5740-e081-47e9-a15b-97c007d43e1b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:09.952667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 58/112] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:09.956194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fedbf-ac1b-4ea6-b1cb-3dc5c30b18a7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:09.956693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fedbf-ac1b-4ea6-b1cb-3dc5c30b18a7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:09.957424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fedbf-ac1b-4ea6-b1cb-3dc5c30b18a7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:09.964872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 59/113] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:09.964872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fedbf-ac1b-4ea6-b1cb-3dc5c30b18a7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:09.981936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:09.984013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e19a0b-f7da-4961-9c93-ef616c14966e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.078889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.100543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.108121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fedbf-ac1b-4ea6-b1cb-3dc5c30b18a7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.110365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 55/114] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:10.114445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99940-b077-4ea2-854e-86f55b5deabd None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.114886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99940-b077-4ea2-854e-86f55b5deabd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.115168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99940-b077-4ea2-854e-86f55b5deabd None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.116558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 56/115] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:10.120487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa99940-b077-4ea2-854e-86f55b5deabd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.150958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.151604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.151887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.152161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.153325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.154781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9ca16-7312-4153-9ad1-177747f68eb4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.157092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 60/116] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:09 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:10.158199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.158491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.158771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.177578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:10.178770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:10.233000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa99940-b077-4ea2-854e-86f55b5deabd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.251693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 61/117] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:10.258139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.271466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.272105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.272383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.272655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.285704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e5760ca9-1b27-4825-932a-9c95b14ce01d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:10.290816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:10.291677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:10.292826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 57/118] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:10.293743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.293743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.293743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.313789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:10.314996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:10.346531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.377803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.386588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bbcc3-7fd6-441b-9d77-69fe6b9221a3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.389188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 58/119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:10.398050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.432028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.433122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.435024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.435519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.437599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.441115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.445269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 62/120] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:10.446584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.562956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1643ace-3cf0-4dec-a93f-2d6092ab5448 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.587770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301eda1-40cc-46eb-bdce-bf5748be2c89 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.602952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.604246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.604777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.605218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.606651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.609070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.611407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 59/121] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:10.617455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.633125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.633853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.634540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.634977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.635804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:10.636375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.638709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.650343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 63/122] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:10.651393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.664386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.683655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.745599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225009-bf87-46f6-9641-874cfebc54d0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.769252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75cbcf-1784-46bd-a695-3a4d2863537c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.783010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.783795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.784079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.784735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.785506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=54e3afcc046c4c92837a47cd92b1aa2c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:10.785817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.787255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.793409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 60/123] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/role_assignments?user.id=54e3afcc046c4c92837a47cd92b1aa2c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=27f0b0d326d347358345cdc266198dbc => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:10.793992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.818394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.820253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.820929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.821516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.838605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:10.848542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:10.849487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:10.850233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:10.852479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 64/124] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:10.853796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:10.915876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.920742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d88416-2ffc-4764-b6a5-cce90aefcd8b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:10.963790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:10.964703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:10.965156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:10.965581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/86e8ac28358642158fd74d784c25c90c/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:10.968537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, user_id=86e8ac28358642158fd74d784c25c90c, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:10.970828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:10.980230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:10.992204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 61/125] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:10 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/86e8ac28358642158fd74d784c25c90c/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:10.998643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.005065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2c9a-6ed5-418b-877f-23a24d8b35dd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.095568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.096138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.096419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.096694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.098112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.101644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.103705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 65/126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.104629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.222950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094373fc-b4b3-4910-94ac-0611fdd127ae None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.271445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.272084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.272345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.272616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.274006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca19c9-f397-457f-8c58-722aad37bf35 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.284139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:11.289778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:11.290632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:11.291385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 62/127] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:11.292087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.325274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:11.334484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.335109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.335393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.335671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.336348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:11.336656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.342447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.371158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 66/128] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.371768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.404688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:11.434121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe5e9-55c8-4701-bae0-53be97f24182 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.470726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.471371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.471661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.471926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] PATH_INFO: `/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.473115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a8c1b628c11460c88e0f5b699d2b66e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.474571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.476429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 63/129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.477006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.567977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:11.583504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462717-cbe8-4f1b-9168-4e8c4f26ebe1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.607022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59d01-2b3e-461d-8e8e-a9316eaeab3e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.620281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.625854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.625854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.625854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.628966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.628966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.629892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 64/130] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.632077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.681849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.682631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.682913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.683188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.683885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:11.684193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.685886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.708898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 67/131] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.708898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.790056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d5de-4454-4fd6-b2fd-7b75a57529fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.848281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.848961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.849243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.849512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] PATH_INFO: `/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e/users/86e8ac28358642158fd74d784c25c90c/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.851782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a8c1b628c11460c88e0f5b699d2b66e, user_id=86e8ac28358642158fd74d784c25c90c, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.853300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:11.854321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.871651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 65/132] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:11 2022] PUT /identity/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e/users/86e8ac28358642158fd74d784c25c90c/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:11.872762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.883816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488abd-f841-4707-83d6-0a1cadfccc4c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:11.892512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:11.947526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:11.948177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.948460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.948787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.949960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:11.951765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd5f25-879d-46bf-9116-c7db9cd51d6f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:11.962755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 68/133] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:11.964145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5354-fe59-4911-b354-f5cfac19229f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:11.964436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5354-fe59-4911-b354-f5cfac19229f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:11.964815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5354-fe59-4911-b354-f5cfac19229f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:11.966046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 69/134] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:11.969083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc5354-fe59-4911-b354-f5cfac19229f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:11.987560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7503-7977-4df8-9ebb-a67a8400a2b2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.032472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.033167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.033445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.033719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.036165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.037673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349f000-2a12-4fae-baf8-748bd1e6a25e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:12.039364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 66/135] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:11 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:12.040558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.044939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.045237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.090209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:12.091440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:12.178260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc5354-fe59-4911-b354-f5cfac19229f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.196881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 67/136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:12.197600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:12.215825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:12.221869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.222717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.222998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.223370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/955dc67368494986a8ad78d63a857238/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.226462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, user_id=955dc67368494986a8ad78d63a857238, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.228105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:12.242586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 70/137] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:11 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/955dc67368494986a8ad78d63a857238/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:12.323297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1db7ef-c004-46d9-b65d-97c875a701bf None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.364757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.365660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.365945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.366230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.367505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.369095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:12.370780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 68/138] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:12 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:12.377754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:12.382896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:12.513941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0307c37-c7cb-4879-a294-6720c47b57f1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.559823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.560517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.560801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.561111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/86e8ac28358642158fd74d784c25c90c/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.564439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, user_id=86e8ac28358642158fd74d784c25c90c, role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.565925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:12.586389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 71/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:12 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/86e8ac28358642158fd74d784c25c90c/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:12.607577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:12.708888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:12.735284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a5ef-c4aa-427b-8d5a-21c88ffda2e9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.781012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.781666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.781945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.782218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.783376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.784830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6a595-6ea7-41ba-b995-cca2cef39ac5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:12.786766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 69/140] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:12.792013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:12.895336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:12.943346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63754d0b-4491-426e-b392-e952f9a5abae None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:12.992275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:12.993288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:12.993781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:12.994249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:12.995131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:12.996871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59af7a-9899-4786-97b0-951765001305 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:13.024654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 72/141] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:12 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:13.741869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee04ca-f47d-4e8f-b375-d783f42614c2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:13.742845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee04ca-f47d-4e8f-b375-d783f42614c2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:13.743338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee04ca-f47d-4e8f-b375-d783f42614c2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:13.744889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 70/142] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:13.751562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396617f2-78fe-4b00-9f3c-7e7385102caf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:13.752390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396617f2-78fe-4b00-9f3c-7e7385102caf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:13.752878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396617f2-78fe-4b00-9f3c-7e7385102caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:13.775087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-396617f2-78fe-4b00-9f3c-7e7385102caf None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:13.776598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-396617f2-78fe-4b00-9f3c-7e7385102caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:13.865050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 73/143] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:13.981208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a0ff-f68d-4774-b731-a35df138fe46 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:13.981849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a0ff-f68d-4774-b731-a35df138fe46 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:13.982132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a0ff-f68d-4774-b731-a35df138fe46 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:13.983532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 71/144] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:13.989145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:13.989663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:13.990135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:14.012399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:14.013942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:14.066883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:14.094254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 74/145] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:14.287583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1a0ff-f68d-4774-b731-a35df138fe46 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:14.401550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1a0ff-f68d-4774-b731-a35df138fe46 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:14.448552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:14.449247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:14.449528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:14.449803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:14.462495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:14.468091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:14.470364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:14.472189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:14.476029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:14.593505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75d592-a254-44ed-953a-c6e3ff8ca12f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:14.630751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:14.631419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:14.631644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:14.631972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:14.632644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:14.632965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:14.634402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:14.649116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 75/147] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:14.654898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:14.666718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:14.703005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:14.783466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:14.828542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c959-d93b-4a80-98c5-de2cfd539014 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:14.888409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b302af-fd74-40c2-bfe9-7a16fec39b53 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:14.897760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:14.898680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:14.899122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:14.900006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] PATH_INFO: `/v3/users/54e3afcc046c4c92837a47cd92b1aa2c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:14.901812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=54e3afcc046c4c92837a47cd92b1aa2c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:14.903714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:14.906082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 73/148] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/users/54e3afcc046c4c92837a47cd92b1aa2c => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:14.914422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:14.935389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:14.936838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:14.937302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:14.937869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:14.958679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:14.966518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:14.967614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:14.968508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:14.971227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 76/149] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:14.977649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:15.070639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:15.128155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9529fb-9304-4164-a9e8-45f57593f2a1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:15.185036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b093d0f-67c7-4e9b-a43d-d4e06df57ccb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:15.205310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:15.206243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:15.206956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.207176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.207442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.208602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:15.210091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:15.216701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 74/150] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:15.217850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:15.258583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:15.259281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.259561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.259836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.260578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:15.260883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:15.262363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:15.278500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 77/151] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:15.284775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:15.421508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaff836-32c9-4fb1-91a6-75108bac7796 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:15.472032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:15.473012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.473429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.473912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/54e3afcc046c4c92837a47cd92b1aa2c/roles/27f0b0d326d347358345cdc266198dbc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.476394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, user_id=54e3afcc046c4c92837a47cd92b1aa2c, role_id=27f0b0d326d347358345cdc266198dbc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:15.477995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:15.480200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc75ba-cb53-4604-9734-726a897d117a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:15.492235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 75/152] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:15 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/users/54e3afcc046c4c92837a47cd92b1aa2c/roles/27f0b0d326d347358345cdc266198dbc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:10:15.545742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:15.546437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.546705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.546974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.548249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:15.549784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883140ac-a318-41a7-b5ef-c18271213deb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:15.551461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 78/153] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:15 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:15.557812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:15.563519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:15.690694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c407472-db9d-4daa-b148-8708fd1e89ae None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:15.706019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371995-43fe-47a9-8580-e23b78683735 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.706531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371995-43fe-47a9-8580-e23b78683735 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.706968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371995-43fe-47a9-8580-e23b78683735 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.712799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 79/154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:15.714545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594c03-4745-4fd4-9bae-c1354df6b262 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.715051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594c03-4745-4fd4-9bae-c1354df6b262 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.715530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594c03-4745-4fd4-9bae-c1354df6b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.728932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:15.730003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.730496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.730956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.762063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-18594c03-4745-4fd4-9bae-c1354df6b262 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:15.763423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-18594c03-4745-4fd4-9bae-c1354df6b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:15.808143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:15.809780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:15.829371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:15.845967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:15.924456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 76/155] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:15.930277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:15.958300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 80/156] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:15.960825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4d516-38bb-40b3-aa38-2ad124c01b48 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.961354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4d516-38bb-40b3-aa38-2ad124c01b48 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.961838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4d516-38bb-40b3-aa38-2ad124c01b48 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.964698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 81/157] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:15.969255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:15.969777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:15.970272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:15.992893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:15.994798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:16.079962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 82/158] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:16.104784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb20d-9853-4276-b5cd-28092c680116 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:16.115604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:16.169649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:16.170299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:16.171143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:16.171542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:16.173004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:16.174560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:16.176207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 77/159] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:15 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:16.183844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:16.305711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:16.320053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7814bd-f0ee-4f6c-8e75-30d369e6720f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:16.360724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:16.361356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:16.361627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:16.361891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:16.362609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:16.362911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:16.364360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:16.373517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 83/160] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:16 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:16.506610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:16.637574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104062-6641-44c1-b8d0-255ff5a0761b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:16.681734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:16.682395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:16.682676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:16.682949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:16.688745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:16.713940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:16.719581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:16.720438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:16.721698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 78/161] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:16.729033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:16.797295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:16.900685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8635b-b7c1-4eb4-aa07-497e7da4d3c5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:16.968575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:16.969505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:16.969919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:16.970321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:16.971041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65148b0b-2961-46e1-89f7-a06e6757be56 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:16.987202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:16.993043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:16.994093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:16.995033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:16.996847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 84/162] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:17.003722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:17.031874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:17.055063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.055937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.056213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.056484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.057178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:17.057487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.058868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e409d7d-324f-4b18-8e21-92ae1875e10c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:17.076245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 79/163] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:17.077882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1a3b-bac0-4b2d-915d-a891703da664 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.078173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1a3b-bac0-4b2d-915d-a891703da664 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.078444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1a3b-bac0-4b2d-915d-a891703da664 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.080492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 80/164] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:17.086059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a1a3b-bac0-4b2d-915d-a891703da664 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:17.149112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61c36c-080b-41e9-a36a-f63142d5605c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:17.243659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.244559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.245054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.245502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.246348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:17.246847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.248421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8dfd5-e1e3-4918-bbef-b22dc98e8823 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:17.268624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 85/165] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:17.270488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.271026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.271670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.291342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a1a3b-bac0-4b2d-915d-a891703da664 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:17.304974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:17.318031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:17.319572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:17.346069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.346826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.347112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.347388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] PATH_INFO: `/v3/users/955dc67368494986a8ad78d63a857238` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.351900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] RBAC: Authorizing `identity:get_user(user_id=955dc67368494986a8ad78d63a857238)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.351900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:17.352110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 81/166] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:17 2022] GET /identity/v3/users/955dc67368494986a8ad78d63a857238 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:17.352868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:17.531115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 86/167] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:17.531990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:17.549269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:17.558888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4f16-3793-49de-891c-cebbd5bf4051 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:17.649710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.650362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.650642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.650919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.652106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.653693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:17.655702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 82/168] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:17 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:17.661385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:17.707491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174123e-df26-48fd-86cc-57c45f8b44b4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:17.752520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:17.773636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.774263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.774602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.774879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.839609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7a655-460c-44c9-ae78-c15f330c3b1b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:17.883055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:17.883661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:17.883947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:17.884214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:17.949733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.951249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:17.973307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:17.992106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:17.993641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:18.050157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:18.152616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 83/169] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:18.153268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.198528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 87/170] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:18.199383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.322518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cdff3-d7d3-4887-b59f-8700a07f3fa7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:18.374556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:18.385810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:18.386440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.386712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.386975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.389252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb34d-74e6-4a31-b95e-ae8a4a47e693 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:18.407756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:18.413012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:18.418065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:18.419028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:18.419881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 84/171] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:18.421147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.460079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:18.460709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.460998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.461268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] PATH_INFO: `/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.462574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] RBAC: Authorizing `identity:get_project(project_id=5a8c1b628c11460c88e0f5b699d2b66e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:18.464937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea05a9-80ac-4d8d-b22b-d807d49f9428 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:18.466569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 88/172] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/projects/5a8c1b628c11460c88e0f5b699d2b66e => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:18.468216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902726-0ab8-4103-b407-38ddcd45c5d6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.470912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902726-0ab8-4103-b407-38ddcd45c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.472017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902726-0ab8-4103-b407-38ddcd45c5d6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.476292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 89/173] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:18.478244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff902726-0ab8-4103-b407-38ddcd45c5d6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.572333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d980bc5-cb15-4726-aa30-f16359a89656 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:18.606535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff902726-0ab8-4103-b407-38ddcd45c5d6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:18.617529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:18.618811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.619591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.620071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.621673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:18.623306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:18.625373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 85/174] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:18.626330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.647420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:18.648465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.648979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.649432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.650378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:18.650873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:18.652589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334573a2-2fb8-49fb-bb00-89e670003587 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:18.668005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 90/175] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:18.669578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678e6e-38e0-4983-93bd-a24bbed76420 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.670010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678e6e-38e0-4983-93bd-a24bbed76420 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.670460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678e6e-38e0-4983-93bd-a24bbed76420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.691550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-87678e6e-38e0-4983-93bd-a24bbed76420 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:18.692185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:18.694686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87678e6e-38e0-4983-93bd-a24bbed76420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:18.731551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:18.745436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb261b-3a35-4c2e-afc3-416593970b22 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 91/176] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba794e6-6c6b-426c-b29b-8b4e6c48d20e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba794e6-6c6b-426c-b29b-8b4e6c48d20e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba794e6-6c6b-426c-b29b-8b4e6c48d20e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 92/177] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:18.788774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba794e6-6c6b-426c-b29b-8b4e6c48d20e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.801542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:18.802703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:18.803144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:18.803610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:18.805774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=5a8c1b628c11460c88e0f5b699d2b66e, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:18.806297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:18.814704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:18.827747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 86/178] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=5a8c1b628c11460c88e0f5b699d2b66e&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:18.828909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:18.970554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba794e6-6c6b-426c-b29b-8b4e6c48d20e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.012186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.013074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.013517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.013931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.015295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=955dc67368494986a8ad78d63a857238) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:19.015784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:19.017538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d402d96-0c79-4743-a8eb-2f2fc6ddec08 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:19.025023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 93/179] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/role_assignments?user.id=955dc67368494986a8ad78d63a857238&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:19.026662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.027114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.027525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.028458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ae72-4447-466f-b019-fa3b50e9827e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.050264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:19.051738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:19.056748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.078342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.078950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.079227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.079497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] PATH_INFO: `/v3/users/86e8ac28358642158fd74d784c25c90c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.085890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] RBAC: Authorizing `identity:get_user(user_id=86e8ac28358642158fd74d784c25c90c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:19.087403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649ec9-3f32-4a61-a48c-59cdc1071b65 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:19.089364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 87/180] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:18 2022] GET /identity/v3/users/86e8ac28358642158fd74d784c25c90c => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:19.090402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1ede-71f4-4abb-a524-7d66a7ce32a0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.090597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.090719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1ede-71f4-4abb-a524-7d66a7ce32a0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.090931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a1ede-71f4-4abb-a524-7d66a7ce32a0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.096970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 88/181] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:19.098818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fd980-2819-4c2c-8ff5-e2f313b8468e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.099102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fd980-2819-4c2c-8ff5-e2f313b8468e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.099371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fd980-2819-4c2c-8ff5-e2f313b8468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.141040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f46fd980-2819-4c2c-8ff5-e2f313b8468e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:19.145491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f46fd980-2819-4c2c-8ff5-e2f313b8468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:19.200016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 94/182] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:19.201209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.277017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 89/183] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:19.282389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86992-f350-4581-8556-70a2b91f471e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.282913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86992-f350-4581-8556-70a2b91f471e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.283372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86992-f350-4581-8556-70a2b91f471e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.285185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 90/184] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:19.290894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.291446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.291917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.312527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:19.314217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:19.384006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5cdcf-24f5-479f-9807-682807351ba0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.386719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 91/185] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:19.431224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.446318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.447286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.447589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.447865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.493447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.517324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.588983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccd9a3-1ca1-41ec-a3ab-00a45a96350f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.594937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:19.596787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:19.642522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.643353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.643632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.643899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.659078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:19.662168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:19.663037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 92/186] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:19.665159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.734510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 95/187] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:19.739954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.775725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be68745-3a30-4728-9a42-e8519cf4178c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.787283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.802227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:19.813114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.813739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.814011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.814276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.824311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:19.829724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:19.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 93/188] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:19.831547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.845310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db9606-e710-43be-bee7-a1d1f1dcb0e0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.878516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.879135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.879412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.879689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.880792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:19.882259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:19.883819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 96/189] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:19.884661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.931994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da8a5-6a12-4df1-9712-80aff0de6091 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:19.965574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:19.966168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:19.966441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:19.966716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:19.967398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:19.967770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:19.969176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:19.977034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 94/190] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:19.977825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:19.984441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f936-f98d-4306-826e-61cd95fe1f18 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.010104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.012738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.019386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.020003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.020274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.020537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.021666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.023090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.024683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 97/191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.025469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.083866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd8a9-b826-4664-afa9-c667752304ce None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.117167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.117831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.118207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.118484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.122812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772faa-91be-4555-9b59-17e1b88cd2fa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.129245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:20.134530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:20.137001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 95/192] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:20.137859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.158871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.159597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.159979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.160342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.161040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:20.161347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.162740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.171296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 98/193] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.171296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.229473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.233610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.242675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03bab-0cf2-4c3e-b372-7b683f22e1c6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.271037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f991c-214e-43e7-8875-c5f14f3c27c8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.276233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.276839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.277140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.277411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.278070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.279491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.306341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 96/194] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:20 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:20.307271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.309563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.310236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.310513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.310782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.327624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.330812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.336187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 99/195] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.336187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.443422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02310-b108-4c29-8111-879fb89a37da None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.457664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f22b-053f-4f72-a44e-5494c91affe7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.463150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.482626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.484176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.484456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.484732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.485461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:20.485765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.487143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.495791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 100/196] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.497954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.499639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.500270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.500553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.500826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.501599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=86e8ac28358642158fd74d784c25c90c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:20.501910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.503370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.509996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 97/197] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/role_assignments?user.id=86e8ac28358642158fd74d784c25c90c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.510913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.530611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.645380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c96c-8499-43ae-82fa-7d091eba36ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.661680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40695d-e47f-4417-9b37-9fbc3195a77c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.692688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.693395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.693728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.693949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.695828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.698776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.700194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 101/198] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.701177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.714094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.714708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.714986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.715253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.764607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.807680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.809243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.829960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:20.838881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980cfd0-fcda-4b3f-ab60-13b4553b3a66 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:20.875895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:20.876536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:20.876814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:20.877104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] PATH_INFO: `/v3/users/54e3afcc046c4c92837a47cd92b1aa2c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:20.878265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] RBAC: Authorizing `identity:get_user(user_id=54e3afcc046c4c92837a47cd92b1aa2c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:20.879933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:20.882010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 102/199] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/users/54e3afcc046c4c92837a47cd92b1aa2c => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:20.883095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:20.931346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 98/200] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:20.932198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.013291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0216e-8169-4147-a152-661e948df579 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.039449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dba7e7-36e9-4694-a894-e355a5f84489 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.053425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.054010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.059758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.060808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.069782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.069782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.077705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.078570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.078570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.078845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.152859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.153497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.154348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.154608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.273390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 99/201] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:21.274498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.291535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.305031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 103/202] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:21.306280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.325074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.396775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f9ffe-82cb-4bd1-9271-ec97cf465605 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.414734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8ac12-1e71-4626-bd55-5773669a6122 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.433904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.434629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.434976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.435312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.436712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.438295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.445367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 100/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:21.446459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.450845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.451477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.451815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.452086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.453380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.454834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.456544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 104/204] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:21.457344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.539364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.546069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:21.563125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd434ea-caec-46b9-b96d-9ea1edff8e55 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.564335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765e96-98e8-4d47-be58-8638b79d759d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.597461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.598059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.598330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.598599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.599316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=f69a8bc8475d48759c452eae3842d6e5) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:21.599599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.601364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.602795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.603518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.603796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.604171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.605319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.606850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.607236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 105/205] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/role_assignments?user.id=f69a8bc8475d48759c452eae3842d6e5&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:21.608160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.609343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 101/206] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:21.617056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:21.729429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242ffe7-d916-401c-b689-f91c5d7dbd75 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.765979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.766641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.766918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.767235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.767972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba, user.id=54e3afcc046c4c92837a47cd92b1aa2c) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:21.768275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.769784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900c5a4-86bc-4527-b813-f621b21fda41 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.776271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 106/207] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/role_assignments?user.id=54e3afcc046c4c92837a47cd92b1aa2c&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=27f0b0d326d347358345cdc266198dbc => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:21.822367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d1754-818c-493f-8587-c1a9c31ec12c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:21.880719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:21.881375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:21.881650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:21.881918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:21.882660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=f69a8bc8475d48759c452eae3842d6e5) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:21.882964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:21.884411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5231f4c-d8f3-4112-91f7-ce74774dafd7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:21.897117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 102/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:21 2022] GET /identity/v3/role_assignments?user.id=f69a8bc8475d48759c452eae3842d6e5&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:22.841156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e4bc-301c-4a7d-9e5e-acaf17425832 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:22.842298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e4bc-301c-4a7d-9e5e-acaf17425832 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:22.842782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e4bc-301c-4a7d-9e5e-acaf17425832 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:22.844760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 107/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:22.850838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1a4df-d0c5-48b7-aff5-ad10d8dfd78c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:22.851798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1a4df-d0c5-48b7-aff5-ad10d8dfd78c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:22.852267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1a4df-d0c5-48b7-aff5-ad10d8dfd78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:22.871279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dbc1a4df-d0c5-48b7-aff5-ad10d8dfd78c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:22.872770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dbc1a4df-d0c5-48b7-aff5-ad10d8dfd78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:22.881175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:22.943306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 103/210] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:23.068264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1d3-1843-4a09-86e9-32419333870c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.069102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1d3-1843-4a09-86e9-32419333870c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.069600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1d3-1843-4a09-86e9-32419333870c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.071428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 108/211] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:23.077680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.077972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.078244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.097061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:23.098241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:23.167797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 104/212] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:23.250995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74b1d3-1843-4a09-86e9-32419333870c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.267943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:23.363863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74b1d3-1843-4a09-86e9-32419333870c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.400312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.400999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.401284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.401574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.412504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:23.417756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:23.418595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:23.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 109/213] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:23.425939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.445368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.507963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:23.544853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa63e-24a2-4896-920d-9a5227358788 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.553329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:23.566421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278709-1647-49d1-9b7b-2b1f3318023d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.589359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.589976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.590254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.590668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.591487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:23.591789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:23.594239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:23.605259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 105/214] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:23.605969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.607239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.607517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.608431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.613827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:23.615378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:23.617104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 110/215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:23.620994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.622455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.741098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de69209-6675-49d7-a045-54302518f2a6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.760305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0600-02d2-4107-8fce-56ee1e0f33f8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.778301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.779163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.780041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.780479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] PATH_INFO: `/v3/users/b20282cbc72e426bac89b446961d61da` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.782736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:23.799403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] RBAC: Authorizing `identity:get_user(user_id=b20282cbc72e426bac89b446961d61da)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:23.801229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:23.802263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.803033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.808025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.808142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 106/216] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:23 2022] GET /identity/v3/users/b20282cbc72e426bac89b446961d61da => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:23.808236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:23.808236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:23.808236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae88af-c134-453c-92cf-8bcb5b071fc2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:23.809493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:23.822255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:23.840573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 111/217] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:23 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:10:23.940115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feef91-97af-41ac-a965-638d796f0d6c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:23.981377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:23.982356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:23.982869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:23.983359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.057994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:24.069292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:24.071017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0693b-3177-483e-ae84-d78bd1c50c02 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:24.142208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f875-438f-49a3-8344-2516b80ddbb8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.142728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f875-438f-49a3-8344-2516b80ddbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.142728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f875-438f-49a3-8344-2516b80ddbb8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.144220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 112/218] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:24.150150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381fff-ccdc-4236-a2f3-4f9b3f582330 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.150428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381fff-ccdc-4236-a2f3-4f9b3f582330 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.150694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381fff-ccdc-4236-a2f3-4f9b3f582330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.176205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5c381fff-ccdc-4236-a2f3-4f9b3f582330 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:24.177793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5c381fff-ccdc-4236-a2f3-4f9b3f582330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:24.186723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 107/219] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:10:24.188305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949575-1472-4d2c-936f-904afdf834b6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.188650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949575-1472-4d2c-936f-904afdf834b6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.188931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949575-1472-4d2c-936f-904afdf834b6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.190580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 108/220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:24.195464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9949575-1472-4d2c-936f-904afdf834b6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:24.268760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 113/221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:24.275005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.275655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.276030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.303452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:24.304925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:24.312066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:24.330319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9949575-1472-4d2c-936f-904afdf834b6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:24.397907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:24.398817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.399235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.399645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.402099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:24.403756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:24.409688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 109/222] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:24 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:24.414105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:24.438595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 114/223] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:24.547756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b904-6519-4576-ad25-f0f04dcdeb7c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:24.602718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:24.603330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.603607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.603880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.604611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=b20282cbc72e426bac89b446961d61da) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:24.604919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:24.606402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:24.610086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:24.621033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 110/224] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:24 2022] GET /identity/v3/role_assignments?user.id=b20282cbc72e426bac89b446961d61da&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:24.707882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:24.766074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483ffd9-2a12-4862-9620-b3db5982da3e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:24.806807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:24.813369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:24.814060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:24.814340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:24.814621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:24.840823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:24.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:24.847488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:24.848268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:24.850985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 115/225] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:24.861375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:24.862410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:24.992412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0f0a-05f7-4f78-a808-98b1ac7e99e2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.053900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.055951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.057527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.058138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.067280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0926-4f7a-4b46-82b8-8df20a31848b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.071166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:25.077677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:25.078747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:25.079515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:25.081845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 111/226] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:25.088116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.099069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:25.136297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.136910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.137206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.137473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.138141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:25.138440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.140539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.156478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 116/227] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:25.165942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.194333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:25.219247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039547-2816-453c-a46a-e1d83c0fda1c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.295948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.296632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.296915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.297278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.297958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:25.298263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.299681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.315872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633518-a08c-433f-975d-e00e65c7c999 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.323231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 112/228] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:25.329852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.356937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.357623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.357928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.358197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.361562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.363014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.364617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 117/229] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:25 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:25.370398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.423514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:25.481582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55db444-d94e-41e1-8823-76b5252b74a7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.518435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.519164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.519434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.519700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.520814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.522587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.524212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 118/230] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:25 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:25.529805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.615734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae418a-d716-47a0-9836-58e9b0673285 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.640466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cabbdb-f5cc-4dae-b1ed-2ec077c7bd0c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.669288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:25.684710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.685660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.685660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.685957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.687633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.690851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.690851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 113/231] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:25 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:25.704837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:25.706133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.706841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.707116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.707385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/f69a8bc8475d48759c452eae3842d6e5/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.709960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, user_id=f69a8bc8475d48759c452eae3842d6e5, role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.711797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.724424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:25.728993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 119/232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:25 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/f69a8bc8475d48759c452eae3842d6e5/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:10:25.853035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5556e-aec5-4f93-bad9-e75495ecd7a9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:25.898653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:25.899493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:25.899771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:25.900038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:25.901191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:25.902682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c69d-5bb7-4c23-8d55-dde56db877a0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:25.904586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 114/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:25 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:25.910792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:26.024709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f92607-8e6f-47e5-b035-d38c120778b1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:26.061648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:26.062315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:26.062592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:26.062860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/f69a8bc8475d48759c452eae3842d6e5/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:26.064883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, user_id=f69a8bc8475d48759c452eae3842d6e5, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:26.066543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0d2cd-1be0-4dbd-b915-646af23e0045 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:26.081742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 120/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:25 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/f69a8bc8475d48759c452eae3842d6e5/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:26.515509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e449-405b-4103-8714-07c4dc5cfd19 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:26.516065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e449-405b-4103-8714-07c4dc5cfd19 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:26.516065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e449-405b-4103-8714-07c4dc5cfd19 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:26.517393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 115/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:26.523327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474e31-45cf-4b25-8269-7c217868a4c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:26.523831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474e31-45cf-4b25-8269-7c217868a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:26.523949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474e31-45cf-4b25-8269-7c217868a4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:26.547739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-49474e31-45cf-4b25-8269-7c217868a4c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:26.549010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49474e31-45cf-4b25-8269-7c217868a4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:26.579174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:26.624506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 121/236] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:26.705706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709d62-6ab0-4101-bb6d-8217ffd3572f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:26.706096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709d62-6ab0-4101-bb6d-8217ffd3572f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:26.706464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709d62-6ab0-4101-bb6d-8217ffd3572f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:26.708361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 116/237] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:26.713642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:26.713924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:26.714246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:26.734410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:26.735601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:26.805270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 122/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:27.069612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b709d62-6ab0-4101-bb6d-8217ffd3572f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.109740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.205753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:27.227274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b709d62-6ab0-4101-bb6d-8217ffd3572f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.273085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.274184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.274614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.275032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.288119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:27.293488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:27.294568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:27.295850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:27.297905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 117/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:27.307642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.349421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:27.359885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3069729-d5fc-464f-97cf-a8e561b45830 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.404269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.405144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.405976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.406393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.407916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:27.409946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6523d-f217-44ef-bc7f-42c614e2e022 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:27.411984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 123/240] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:27.413352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e749d8-f848-497d-a238-47fdd5ddc735 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.413766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e749d8-f848-497d-a238-47fdd5ddc735 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.414851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e749d8-f848-497d-a238-47fdd5ddc735 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.416573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 124/241] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:27.417631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e749d8-f848-497d-a238-47fdd5ddc735 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.428007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a4c3c-bde3-4e93-bfd8-4f60d1603b0a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.467726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.468474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.468855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.469255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.470020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:27.470425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:27.472161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bedeee-ce76-4c96-b680-0227890eca0e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:27.482796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 118/242] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:27.484503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.484908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.485297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.510712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:27.512534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:27.514774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:27.557469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e749d8-f848-497d-a238-47fdd5ddc735 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.601275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 119/243] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:27.602153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.605046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.606917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.607414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.608625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.609878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:27.611577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:27.649019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 125/244] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:27 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:27.719748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95181-4520-4e0f-b50f-3dc13df28c6c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.756996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.757639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.757916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.758186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] PATH_INFO: `/v3/users/6e6003ecb1b545d88576456f96cb0f40` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:27.761677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:27.775876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] RBAC: Authorizing `identity:get_user(user_id=6e6003ecb1b545d88576456f96cb0f40)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:27.777880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:27.780170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 120/245] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:27 2022] GET /identity/v3/users/6e6003ecb1b545d88576456f96cb0f40 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:27.786493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:27.822477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:27.933706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880be5-6a31-4443-a30c-21d3668c68e3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:27.973369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:27.974321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:27.974790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:27.975272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.006693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.057328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.059165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.080908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:28.141014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540053-d50a-4f17-874b-eb08b9d376ae None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.182699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 126/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:28.189365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.190532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.191210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.191490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.191805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.206002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:28.210854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:28.211730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:28.212518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:28.217446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 121/247] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:28.220418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.287933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:28.327837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbcdd1-2b77-488d-a43f-f2f4477cd427 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.347722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ef5b8-de7b-4654-b146-2b8df70b6927 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.351823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:28.372584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.374101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.374561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.374919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.376638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.378480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.380245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 127/248] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:28.393756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.406510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.407616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.408129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.408604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.409556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:28.410124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.411781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.429074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 122/249] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:28.431513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.531156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d259ad-d8eb-4b8a-9178-3273234816a6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.586186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.588040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.588497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.588927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.590052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=6e6003ecb1b545d88576456f96cb0f40) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:28.590826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.594325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18baa8-aa81-4f6e-97b9-f87d981074ae None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.597151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d337f-fe15-49e3-868d-d6a934ab7d11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.609093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 128/250] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/role_assignments?user.id=6e6003ecb1b545d88576456f96cb0f40&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=27f0b0d326d347358345cdc266198dbc => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:28.610803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351c119-04bd-45ce-9701-3afb564f4b6e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.611233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351c119-04bd-45ce-9701-3afb564f4b6e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.611641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351c119-04bd-45ce-9701-3afb564f4b6e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.613506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 129/251] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:28.614427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:28.619753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c979-6c69-4462-8585-b79009bb74c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.620207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c979-6c69-4462-8585-b79009bb74c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.620649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c979-6c69-4462-8585-b79009bb74c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.638370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.639387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.639762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.640136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] PATH_INFO: `/v3/users/b20282cbc72e426bac89b446961d61da` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.641435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] RBAC: Authorizing `identity:get_user(user_id=b20282cbc72e426bac89b446961d61da)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.642987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.645052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 123/252] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/users/b20282cbc72e426bac89b446961d61da => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:28.656268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-86a8c979-6c69-4462-8585-b79009bb74c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:28.656812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.658676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-86a8c979-6c69-4462-8585-b79009bb74c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:28.781074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:28.846820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 130/253] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:28.847982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d58e9-316c-40fb-b163-4b9651ce0a80 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:28.885611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:28.886269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.886549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.886822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.888017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:28.889477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4f5a4-f7dd-4064-9627-6341cff7946f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.889928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:28.890058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4f5a4-f7dd-4064-9627-6341cff7946f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.890306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4f5a4-f7dd-4064-9627-6341cff7946f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.891607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 124/254] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:28 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:28.892855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 131/255] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:28.897938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:28.898379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:28.898646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:28.899040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:28.940544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:28.942012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:28.995524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:29.113040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 132/256] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:29.123365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28b08-c71a-449b-ac62-29f7c158de11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:29.174620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:29.175317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:29.175599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:29.175897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/b20282cbc72e426bac89b446961d61da/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:29.177960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, user_id=b20282cbc72e426bac89b446961d61da, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:29.179491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:29.197528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 125/257] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:28 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/b20282cbc72e426bac89b446961d61da/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:29.257285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:29.421188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af444608-a6e2-4697-a163-4b03ffed5885 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:29.434686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:29.470874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:29.471567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:29.471847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:29.472139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:29.485007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:29.489938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:29.490818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:29.491593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 133/258] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:29.497401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:29.567477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:29.689173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e16d01-c500-4d16-bd09-296a354cd2cc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:29.708220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:29.757070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:29.757860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:29.758144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:29.758416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:29.759093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:29.759395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:29.760770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:29.777919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 126/259] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:29.785295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:29.848745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62dc78-a6a7-440d-bf6d-091f1e21cb96 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:29.943096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:29.944016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:29.944320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:29.944595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:29.975062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:29.980013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:29.980939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:29.981791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 134/260] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:29.985897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627839-ad14-4ddd-8fbd-d3659bd9877f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:29.992534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:30.058353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:30.074122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:30.074826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:30.075131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:30.075406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:30.076795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:30.078530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:30.080382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 127/261] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:29 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:30.102792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:30.177751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:30.244010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e945675-1559-4af3-8ace-71d9a75eddd7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:30.291825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d378-0979-4496-84e3-9519d1df8c61 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:30.307722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:30.308334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:30.308614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:30.308888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:30.309586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:30.309889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:30.311305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:30.329968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 135/262] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:30.332594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:30.354805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:30.355424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:30.355707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:30.355979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:30.499986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:30.501412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:30.554409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:30.574057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b70345-1dcb-4589-bbd6-143b76cf8b5a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:30.586545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:30.655066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e248-930c-4173-82d8-be18828925ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:30.656820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e248-930c-4173-82d8-be18828925ac None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:30.657139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e248-930c-4173-82d8-be18828925ac None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:30.657414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e248-930c-4173-82d8-be18828925ac None admin] PATH_INFO: `/v3/users/f69a8bc8475d48759c452eae3842d6e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:30.659530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355e248-930c-4173-82d8-be18828925ac None admin] RBAC: Authorizing `identity:get_user(user_id=f69a8bc8475d48759c452eae3842d6e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:30.660998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355e248-930c-4173-82d8-be18828925ac None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:30.662814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 136/263] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:30 2022] GET /identity/v3/users/f69a8bc8475d48759c452eae3842d6e5 => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:30.677924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e248-930c-4173-82d8-be18828925ac None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:30.750263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 128/264] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:30.757155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:30.876345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e248-930c-4173-82d8-be18828925ac None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:30.950592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:30.951739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:30.952019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:30.952300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.000190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473daf-e9ac-4262-b385-38e0c3a13fd5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:31.022146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:31.082995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:31.083639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.083919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.084191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.085366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:31.086812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:31.088753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 129/265] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:30 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:31.095801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:31.110222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:31.111716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab56f3-d32f-40ab-8176-8067fe318c16 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:31.118989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:31.274491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bc26a-06df-4462-a526-92fd390794b5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:31.283878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 137/266] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:31.284983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fac8-a26e-4468-aeac-c6952e748dcc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.285272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fac8-a26e-4468-aeac-c6952e748dcc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.285545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fac8-a26e-4468-aeac-c6952e748dcc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.287033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 138/267] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:31.297914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4fac8-a26e-4468-aeac-c6952e748dcc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:31.321798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:31.322746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.323227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.323681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.324587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=f69a8bc8475d48759c452eae3842d6e5) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:31.325121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:31.327419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd5369-3f20-4aad-865f-feb0baa650e6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:31.341115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 130/268] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:31 2022] GET /identity/v3/role_assignments?user.id=f69a8bc8475d48759c452eae3842d6e5&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:31.343178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514d45-25ee-471a-aa05-ed31af207170 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.343655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514d45-25ee-471a-aa05-ed31af207170 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.344114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514d45-25ee-471a-aa05-ed31af207170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.367670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cc514d45-25ee-471a-aa05-ed31af207170 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:31.369409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cc514d45-25ee-471a-aa05-ed31af207170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:31.458023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:31.494832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 131/269] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:31.516194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3393-9909-4604-aa13-98195766cdd5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.516842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3393-9909-4604-aa13-98195766cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.517356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3393-9909-4604-aa13-98195766cdd5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.518953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 132/270] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:31.526735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4fac8-a26e-4468-aeac-c6952e748dcc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:31.530305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.530807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.531310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.545402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:31.560366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:31.562624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:31.608974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:31.608974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.608974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.608974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.610215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:31.611711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:31.614392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 139/271] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:31 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:31.619319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:31.665470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 133/272] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:31.766624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31da683-a00f-423e-a9e0-8bcbe5331cfd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:31.793914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:31.802673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:31.803279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.803550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.803825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.804536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=f69a8bc8475d48759c452eae3842d6e5) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:31.804839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:31.806580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a5e5e-414c-4f98-9a4b-ab43a974b769 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:31.813981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 140/273] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:31 2022] GET /identity/v3/role_assignments?user.id=f69a8bc8475d48759c452eae3842d6e5&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:31.949749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:31.959046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cde9-b414-4d31-8a9b-5791ae9a7d57 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.959566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cde9-b414-4d31-8a9b-5791ae9a7d57 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.959670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cde9-b414-4d31-8a9b-5791ae9a7d57 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.961248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 141/274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:31.967823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:31.968126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:31.968364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:31.990912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:31.992123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:32.072533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e2efa-eee6-452e-bd47-17ab93655343 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.106640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 142/275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:32.107698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.114805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.115720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.116190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.116646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.128065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:32.133342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:32.134199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:32.135074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:32.137329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 134/276] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:32.143172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.163075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.306029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8240a-04fc-450a-a108-fec573a2de94 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.308058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f50a-28c1-4a94-ab95-1899186089bf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.339965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.351471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.351471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.351471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.351471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.352771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:32.354293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dde70d-4dbb-4eec-880f-46595bca67f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:32.362704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 135/277] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:32.362891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.362891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.362891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.362891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.363150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff07-ea95-4668-9cd1-f96cb82fb765 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.363232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:32.363305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff07-ea95-4668-9cd1-f96cb82fb765 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.363374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:32.363444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff07-ea95-4668-9cd1-f96cb82fb765 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.363521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9d424-8103-4b27-a8fb-034a065f795a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:32.363600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 136/278] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:32.364649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ff07-ea95-4668-9cd1-f96cb82fb765 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.373878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 143/279] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:32.375124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.375407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.375681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.397113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:32.398356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:32.431283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.467251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 144/280] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:32.468411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.474566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ff07-ea95-4668-9cd1-f96cb82fb765 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.512058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.512661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.512932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.513221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.513893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:32.516007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:32.539893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 137/281] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:10:32 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:32.540728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.595984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f74e-228e-4d54-ad9f-91f5cf7c0c23 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.616795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.653279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.653928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.654202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.654518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] PATH_INFO: `/v3/users/85df460959b843f98045b6523b2d4e45` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.670886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] RBAC: Authorizing `identity:get_user(user_id=85df460959b843f98045b6523b2d4e45)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:32.672597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:32.674545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 145/282] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/users/85df460959b843f98045b6523b2d4e45 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:32.681127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.683446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bacf9-3e05-4d5c-b32e-8cfb1d869b11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.710234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.733978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.735302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.735986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.736399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.752432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:32.758753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:32.759638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:32.760407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 138/283] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:32.761106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.816925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c82d2-407a-4338-a338-76d38b78ab59 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.862436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.862982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.863262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.863537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.864266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, user.id=85df460959b843f98045b6523b2d4e45, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:32.864750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:32.866920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:32.873814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 146/284] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/role_assignments?user.id=85df460959b843f98045b6523b2d4e45&scope.system=all&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:32.875088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:32.893184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee0f8-821b-49f5-93ee-1813bf5439dd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:32.923097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:32.942720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:32.943439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:32.943711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:32.943974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:32.960203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:32.966776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:32.967736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:32.968498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:32.971991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 139/285] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:32.980738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.020822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739af54-de7e-4f2a-ba9c-e53f563df67e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.045749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:33.074054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.076047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.076047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.076047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.076460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:33.076767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.078854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.089394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 147/286] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:33.094972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.121085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870545-5b09-428f-983d-3f6de7fe1452 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.169702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.170504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.170783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.171048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.171733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:33.172039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.174245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.192662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 140/287] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:33.203013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.259619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b513-3d3c-4a6b-a089-15d487535ec7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.261426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:33.311374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.312358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.312854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.313344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] PATH_INFO: `/v3/users/6e6003ecb1b545d88576456f96cb0f40` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.315236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] RBAC: Authorizing `identity:get_user(user_id=6e6003ecb1b545d88576456f96cb0f40)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.316964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.319062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 148/288] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:33 2022] GET /identity/v3/users/6e6003ecb1b545d88576456f96cb0f40 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:33.324366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.404822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:33.412686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49195fe-4c63-4aba-99a6-4d788aa8ba3a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.478462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8266-2a83-4af1-93c8-cc8cd005a422 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.485697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.486347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.486628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.486897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] PATH_INFO: `/v3/users/b20282cbc72e426bac89b446961d61da` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.488196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] RBAC: Authorizing `identity:get_user(user_id=b20282cbc72e426bac89b446961d61da)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.489882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.491767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 141/289] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:33 2022] GET /identity/v3/users/b20282cbc72e426bac89b446961d61da => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:33.498346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.530387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.531136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.531411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.531676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.532837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.538211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.539882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 149/290] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:33 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:33.549002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:33.586288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:33.687641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbfd9-f9a8-4f9d-b08e-8f56298c78d5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.721925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25377d3d-084a-428a-86ee-b09997a3a789 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:33.764783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.765429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.765704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.765979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.789969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:33.790634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:33.790911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:33.791182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/6e6003ecb1b545d88576456f96cb0f40/roles/27f0b0d326d347358345cdc266198dbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:33.793123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, user_id=6e6003ecb1b545d88576456f96cb0f40, role_id=27f0b0d326d347358345cdc266198dbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.794558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.816566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 150/291] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:10:33 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/users/6e6003ecb1b545d88576456f96cb0f40/roles/27f0b0d326d347358345cdc266198dbc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:33.837357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:33.864586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:33.866225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:33.981348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 142/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:33.990272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:34.104542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:34.115704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691effc9-cca1-4416-ba7c-f13167148304 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:34.151039 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:34.151678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:34.151957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:34.152230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:34.153425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:34.155729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23902-772f-4772-a5ad-2098ed2e8aa5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:34.157404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 151/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:33 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:34.163325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:34.233320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:34.267259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae60082-24b6-4e1c-96eb-fd6737af9584 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:34.304485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:34.305118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:34.305654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:34.305926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:34.306645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=b20282cbc72e426bac89b446961d61da) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:34.306951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:34.308376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccaa16-3c32-48e1-a4fb-aeaa29592ca4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:34.314710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 143/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:34 2022] GET /identity/v3/role_assignments?user.id=b20282cbc72e426bac89b446961d61da&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:35.174008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfe37-3a68-432e-b204-1e0d25e9f293 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.174552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfe37-3a68-432e-b204-1e0d25e9f293 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.174552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfe37-3a68-432e-b204-1e0d25e9f293 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.176063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 152/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:35.181758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f17ee1-82e8-41d0-acac-fcdfff93a3c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.182205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f17ee1-82e8-41d0-acac-fcdfff93a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.182375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f17ee1-82e8-41d0-acac-fcdfff93a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.201608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-28f17ee1-82e8-41d0-acac-fcdfff93a3c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:35.202851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-28f17ee1-82e8-41d0-acac-fcdfff93a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:35.277535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 144/296] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:35.339280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65d9b-b20a-4813-8fc9-b202ae5e091b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.339605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65d9b-b20a-4813-8fc9-b202ae5e091b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.339860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65d9b-b20a-4813-8fc9-b202ae5e091b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.341273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 153/297] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:35.346924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.347217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.347507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.366915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:35.368095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:35.394532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72550-b135-43de-a284-5208a0b3155d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.394830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72550-b135-43de-a284-5208a0b3155d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.395082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72550-b135-43de-a284-5208a0b3155d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.396563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 154/298] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:35.402923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.403204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.403481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.421852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:35.423012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:35.439642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:35.462195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 145/299] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:35.503551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 155/300] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:35.555637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:35.663455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5867f-6c1f-4704-b5de-8f4c921517bb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:35.706635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:35.707522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.707989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.708458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.719082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:35.724157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:35.725028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:35.725785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:35.727923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 146/301] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:35.733803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:35.774888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:35.830897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:35.836326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:35.846088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02da6-d301-497c-a1fc-26b90b669e46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:35.883458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:35.884057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.884380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.884478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed398-88f8-4136-b92d-b358f61e9c2d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:35.884650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.885360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:35.885660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:35.887021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:35.895674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 156/302] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:35.896474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:35.926482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:35.927116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:35.927393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:35.927662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:35.928768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:35.930261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:35.932079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 147/303] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:35.932848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.002593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857ecc-6c05-41b9-8b7c-e544de012978 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.037779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.038406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.038680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.038947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.039417 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16e3e9-1947-454e-9439-3c99c32996f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.049245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:36.054484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:36.055582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:36.056454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 157/304] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:36.057144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.064753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.073255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.085732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.086462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.086745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.087020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] PATH_INFO: `/v3/users/d025c822f8714815ac26946df990e26f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.101335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] RBAC: Authorizing `identity:get_user(user_id=d025c822f8714815ac26946df990e26f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.102796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.104463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 148/305] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:35 2022] GET /identity/v3/users/d025c822f8714815ac26946df990e26f => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:36.105268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.169840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14484-66e9-4d86-8609-4112c5b1dd33 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.205584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.206268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.206455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.206719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] PATH_INFO: `/v3/groups` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.207342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.208767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.218210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb09c9-3347-4363-83c8-0c8328d63a88 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.226495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 158/306] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 07:10:36 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:36.227500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.275254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.275838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.276218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.276389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.277262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:36.277611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.279045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.294812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 149/307] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:36.296207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.340157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.348637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.383898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc58ec-3820-44bb-a15b-156b9ccc3b1c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.429082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.430062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.430595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.430873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.431600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, user.id=d025c822f8714815ac26946df990e26f, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:36.431912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.433392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4011f9f-d099-4583-8bb5-fd87a276864e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.434330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c69462-330d-4ebd-92ef-0c24e75ee9fb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.439317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 159/308] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 07:10:36 2022] GET /identity/v3/role_assignments?user.id=d025c822f8714815ac26946df990e26f&scope.system=all&role.id=27f0b0d326d347358345cdc266198dbc => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:36.479351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fca-90d2-4d82-ac5c-87a7224d133e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.479632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fca-90d2-4d82-ac5c-87a7224d133e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.479911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fca-90d2-4d82-ac5c-87a7224d133e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.481979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 160/309] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:36.483822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.484595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.485915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.485915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] PATH_INFO: `/v3/users/85df460959b843f98045b6523b2d4e45` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.486543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] RBAC: Authorizing `identity:get_user(user_id=85df460959b843f98045b6523b2d4e45)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.487683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.487943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.488037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.488217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.494264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 150/310] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:36 2022] GET /identity/v3/users/85df460959b843f98045b6523b2d4e45 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:36.495351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.510093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:36.511298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:36.591059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 161/311] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:36.609296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8437a3d-f33d-4fa7-a8b4-d076c359448a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:36.617140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.646483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:36.647153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:36.647429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:36.647700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] PATH_INFO: `/v3/system/users/85df460959b843f98045b6523b2d4e45/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:36.649150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=85df460959b843f98045b6523b2d4e45, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:36.650535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:36.659919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 151/312] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 07:10:36 2022] PUT /identity/v3/system/users/85df460959b843f98045b6523b2d4e45/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:36.876045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:36.944854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:36.978961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9529434-80a9-4345-81a8-9c566c8b4cbe None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.014967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.015938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.016446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.016904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.027783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:37.033612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:37.034459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:37.035208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:37.037767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 162/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:37.045401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:37.146039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ad780-fd2e-4a71-af67-11fcd68ca296 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.181420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.182031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.182309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.182579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.183249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:37.183553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:37.184971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:37.193962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 152/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:37.199367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:37.308354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b408c-d899-4c2f-98c6-0a17415d3778 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.337797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:37.348064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.348765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.349302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.349794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] PATH_INFO: `/v3/users/6e6003ecb1b545d88576456f96cb0f40` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.351007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] RBAC: Authorizing `identity:get_user(user_id=6e6003ecb1b545d88576456f96cb0f40)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:37.352539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:37.354219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 163/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:37 2022] GET /identity/v3/users/6e6003ecb1b545d88576456f96cb0f40 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:37.361224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:37.395035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:37.472628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbad6ce-0512-4f32-a64e-023997521f4b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.507393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.508013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.508295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.508566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.580415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:37.581923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:37.615915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:37.693482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 153/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:10:37.698640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:37.803456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573ccd-abd2-4d28-89ff-2b2b4a0ea02b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.840707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.841363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.841753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.842166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.843514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:37.845062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24e8af-419b-47a3-afa0-8b4bbadfb6f0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:37.846985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 164/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:37 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:37.852776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:37.954768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e808-f9d8-4b01-9d5e-8f958ac845df None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:37.986840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:37.992268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:37.992869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:37.993245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:37.993518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:37.994288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f, user.id=6e6003ecb1b545d88576456f96cb0f40) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:37.994600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:37.996026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42927957-93fd-4a19-8830-c66a277fc804 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:38.002666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 154/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:10:37 2022] GET /identity/v3/role_assignments?user.id=6e6003ecb1b545d88576456f96cb0f40&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=27f0b0d326d347358345cdc266198dbc => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:38.700556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c63c5-1230-4f5d-848d-a939ef715ddb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.701135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c63c5-1230-4f5d-848d-a939ef715ddb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.701262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c63c5-1230-4f5d-848d-a939ef715ddb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.702723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 165/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:38.707819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a65db-dc3d-4192-8140-7f4d417307eb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.708207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a65db-dc3d-4192-8140-7f4d417307eb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.708362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a65db-dc3d-4192-8140-7f4d417307eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.714995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e870-c376-486d-bbf5-8bcd50b29cc9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.715277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e870-c376-486d-bbf5-8bcd50b29cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.715550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e870-c376-486d-bbf5-8bcd50b29cc9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.717569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 166/320] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:38.722263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdec0-16ca-497f-a4b4-0f657b755479 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.722537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdec0-16ca-497f-a4b4-0f657b755479 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.722803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdec0-16ca-497f-a4b4-0f657b755479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.727146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d72a65db-dc3d-4192-8140-7f4d417307eb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:38.728334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d72a65db-dc3d-4192-8140-7f4d417307eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:38.741116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-903cdec0-16ca-497f-a4b4-0f657b755479 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:38.742293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-903cdec0-16ca-497f-a4b4-0f657b755479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:38.801397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:38.802022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 155/321] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:38.815789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 167/322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:38.933722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490013f-3b1a-4f10-87a4-0fd439413593 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.934027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490013f-3b1a-4f10-87a4-0fd439413593 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.934280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490013f-3b1a-4f10-87a4-0fd439413593 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.937002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 156/323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:38.941916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:38.942281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:38.942629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:38.960703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:38.961923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:39.033117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 168/324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:39.071276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490013f-3b1a-4f10-87a4-0fd439413593 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.088184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.178100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490013f-3b1a-4f10-87a4-0fd439413593 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.188710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868f813-ce21-4d05-8a3d-e1a7d660d648 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.205279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.213151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.217565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.218500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.218850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.219117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.220327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.221788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.223786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.224410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.224682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.224975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.228328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 157/325] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:39.229159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.235906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:39.240690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:39.241587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:39.242387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 169/326] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:39.247375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.333888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2b2d-2707-478a-9962-77ace2b9379d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.352135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894131-9f88-40b6-87a1-3d9eefa79549 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.367459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.368060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.368330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.368593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] PATH_INFO: `/v3/groups` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.369302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.370889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.386261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 158/327] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 07:10:39 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:10:39.387066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.395325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.395956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.396263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.396536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.397332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:39.397636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.399081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.417679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 170/328] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:39.419408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.461637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.476340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.513326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2606-39e0-4a5d-9be9-14898cd416f5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.547092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabbb57-8462-464f-81d2-e08df9c8f8d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.551422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.552067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.552348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.552624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.564916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:39.570618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:39.571483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:39.572239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 159/329] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:39.578673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.589957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.590195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.590474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.590744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] PATH_INFO: `/v3/users/d025c822f8714815ac26946df990e26f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.592530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] RBAC: Authorizing `identity:get_user(user_id=d025c822f8714815ac26946df990e26f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.594056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.595645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 171/330] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/users/d025c822f8714815ac26946df990e26f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:39.601404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.693928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7e72-ddc8-4dbb-a512-32fd10ffc6ef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.713195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.736905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868702fc-f085-4bdf-9b07-468a12a54eac None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.738265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.739057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.740175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.740451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.741155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:39.741484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.742831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.753227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 160/331] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:39.753873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:39.758627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:39.782765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.782765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.782765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.782765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] PATH_INFO: `/v3/system/users/d025c822f8714815ac26946df990e26f/roles/27f0b0d326d347358345cdc266198dbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.783154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d025c822f8714815ac26946df990e26f, role_id=27f0b0d326d347358345cdc266198dbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.784472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.793919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 172/332] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 07:10:39 2022] PUT /identity/v3/system/users/d025c822f8714815ac26946df990e26f/roles/27f0b0d326d347358345cdc266198dbc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:10:39.889389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55d14-184a-48d3-95ed-7814386e54b8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:39.932592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:39.933293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:39.933574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:39.933846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] PATH_INFO: `/v3/users/85df460959b843f98045b6523b2d4e45` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:39.935112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] RBAC: Authorizing `identity:get_user(user_id=85df460959b843f98045b6523b2d4e45)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:39.937037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea359088-140e-4bcb-9482-0874d24a2b7e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:39.938842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 161/333] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/users/85df460959b843f98045b6523b2d4e45 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:39.945035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:40.068495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d04027-951e-42db-8850-fdcc7e5a1273 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:40.112183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:40.112792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:40.113131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:40.113397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:40.114120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] RBAC: Adding query filter params (role.id=e036f4b567c14abab4c3011e4611ab47, user.id=85df460959b843f98045b6523b2d4e45, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:40.114422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:40.115864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabff06-c8cb-44b9-a164-0559c79db271 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:40.130747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 173/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 07:10:39 2022] GET /identity/v3/role_assignments?user.id=85df460959b843f98045b6523b2d4e45&scope.system=all&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:41.836406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872337d9-2812-4519-88d4-dbb17a93eb52 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:41.837447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872337d9-2812-4519-88d4-dbb17a93eb52 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:41.837940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872337d9-2812-4519-88d4-dbb17a93eb52 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:41.839491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 162/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:41.845080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99995c64-66c6-4db5-bfc5-14d7a4d97c1d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:41.845931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99995c64-66c6-4db5-bfc5-14d7a4d97c1d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:41.846407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99995c64-66c6-4db5-bfc5-14d7a4d97c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:41.870019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-99995c64-66c6-4db5-bfc5-14d7a4d97c1d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:41.871548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99995c64-66c6-4db5-bfc5-14d7a4d97c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:41.904236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:41.948178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 174/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:42.135514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875331-cde1-4cb5-8184-7734a312276a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.136214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875331-cde1-4cb5-8184-7734a312276a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.136692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875331-cde1-4cb5-8184-7734a312276a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.138464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 163/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:42.145157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbb9a-c673-45e5-b233-88e6f21b0d41 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.145758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbb9a-c673-45e5-b233-88e6f21b0d41 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.146280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbb9a-c673-45e5-b233-88e6f21b0d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.179411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-15efbb9a-c673-45e5-b233-88e6f21b0d41 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:42.182213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-15efbb9a-c673-45e5-b233-88e6f21b0d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:42.311652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 175/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:42.356891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348d7c0-ccb3-4a9e-a180-6b68b9fc58e4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.358443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348d7c0-ccb3-4a9e-a180-6b68b9fc58e4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.358944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348d7c0-ccb3-4a9e-a180-6b68b9fc58e4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.361022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 164/339] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:42.366664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.367224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.367688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.391108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:42.392726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:42.457771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348d7c0-ccb3-4a9e-a180-6b68b9fc58e4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:42.476106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 176/340] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:42.543684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:42.614061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348d7c0-ccb3-4a9e-a180-6b68b9fc58e4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:42.657706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:42.658273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.658547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.658813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.675810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:42.681583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:42.682511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:42.683737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 165/341] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:42.694221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:42.728271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:42.739485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:42.892456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2648-bd87-478d-ae71-600f82897c70 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:42.896065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dd5cc-7ef2-4e3a-86d7-07ac1432171a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:42.899867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:42.964982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:42.965972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.966501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.966889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.984644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:42.985656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:42.986126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:42.986602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:42.987463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:42.987954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:42.989576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:42.989940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:42.995842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:42.996833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:42.999020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 166/342] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:43.005308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 177/343] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:43.007367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.012728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.165540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:43.177125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3372d4-6f38-40c1-856a-ddc16da30eff None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.188333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d573325-2cff-4cae-b528-36c78d04d675 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.226832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.228462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.228985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.229521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.267075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.268084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.268560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.269262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.271207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:43.271978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:43.274846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:43.295430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 167/344] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:43.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.334484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:43.336429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:43.376367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:43.460805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:43.478987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 178/345] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:43.480124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.557199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ee17-34b4-4e71-8e6b-797281baeca1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.624051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.624679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.624966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.625233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.648998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:43.655124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:43.656134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:43.657368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 168/346] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:43.658060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.668661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50461feb-a38d-4668-a1e8-e778ffbe4591 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.734365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.735347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.736086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.736560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] PATH_INFO: `/v3/users/d025c822f8714815ac26946df990e26f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.738679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] RBAC: Authorizing `identity:get_user(user_id=d025c822f8714815ac26946df990e26f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:43.740866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:43.743100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 179/347] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/users/d025c822f8714815ac26946df990e26f => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:43.744165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.802485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:43.830656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:43.830964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636f98-3e48-44b3-a1c2-39d139932c3d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.880511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.881794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.882707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.883441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.885642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:43.888313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2bf62-e5cb-4526-a310-c95dbb8d2ad1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:43.892109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 169/348] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:43.892581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e18c81-df76-40e5-8445-100195f33d0a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:43.893756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058fa57-373d-4e55-8a8d-7b74a59c2f70 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.894551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058fa57-373d-4e55-8a8d-7b74a59c2f70 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.895236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058fa57-373d-4e55-8a8d-7b74a59c2f70 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.897040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 170/349] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:43.898003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058fa57-373d-4e55-8a8d-7b74a59c2f70 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:43.936856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:43.937571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:43.937821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:43.938090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:43.938868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:43.939106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:43.940647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:43.952114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 180/350] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:43.952995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.043203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058fa57-373d-4e55-8a8d-7b74a59c2f70 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.079402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e5214-6b30-47ce-afc2-212e4ee17fb5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.091439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.092818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.093335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.094213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.095395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] RBAC: Adding query filter params (role.id=27f0b0d326d347358345cdc266198dbc, user.id=d025c822f8714815ac26946df990e26f, scope.system=all) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:44.095884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:44.098266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974c33c-5e69-4b2a-a347-cc21e52993fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:44.102414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.118247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 171/351] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/role_assignments?user.id=d025c822f8714815ac26946df990e26f&scope.system=all&role.id=27f0b0d326d347358345cdc266198dbc => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:44.119658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.120008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.120397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.129782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.130752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.131224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.131669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.145261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:44.146589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:44.149648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:44.151140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:44.153159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 181/352] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:43 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:44.154094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.158844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.232657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 172/353] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:44.233586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.279993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca987d-7f75-49b1-91d4-4039453f9477 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.321477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.322438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.322909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.323388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.345264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7135af-75fd-4704-853a-7a6e977f181f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.399940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.400563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.400840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.401133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.401871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:44.402176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:44.403613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:44.419369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 173/354] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:44 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:44.441308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.462956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:44.464446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72c66-ef01-4a06-940e-d5d66cdee2f0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:44.609802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 182/355] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:44.615974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.633540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.695326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dc8ff-dba7-4a24-9faa-10c10365473f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.695651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dc8ff-dba7-4a24-9faa-10c10365473f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.695854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dc8ff-dba7-4a24-9faa-10c10365473f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.697547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 183/356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:44.706920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.707201 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.707473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.729509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:44.730705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:44.760841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bbe0-c412-401e-8bd7-54d6624dd8f8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.799950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.800578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.800849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.801148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:44.802437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:44.804357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:44.806926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 174/357] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:44 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:44.807706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.813775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 184/358] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:44.814658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:44.848180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.908122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:44.935358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b55fb-5b15-4be6-bca2-a3d703bedd59 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:44.990278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:44.990926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:44.991204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:44.991478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.011261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e9de68b4-2f91-479e-a935-73903f02c74b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:45.018735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:45.019638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:45.020410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:45.026786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 175/359] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:45.026786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5c7f3-0f25-44f7-a4f0-15646ced414d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.026786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5c7f3-0f25-44f7-a4f0-15646ced414d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.026786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5c7f3-0f25-44f7-a4f0-15646ced414d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.027541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 176/360] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:45.028286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5c7f3-0f25-44f7-a4f0-15646ced414d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.049875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46c23-a4c2-4c74-abfc-499d067fff79 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.096671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.097351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.097629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.097903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.101196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:45.102774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c717-b65c-45b0-8c05-7ad98a11968b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:45.104748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 185/361] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:44 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:45.106146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.106430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.106702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.128136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:45.129613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:45.148491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5c7f3-0f25-44f7-a4f0-15646ced414d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.187174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.187696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.187966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.188229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.188886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:45.189210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:45.190559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:45.191950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.203305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 177/362] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:45.204146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.219663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 186/363] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:45.220568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.235621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.334284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e5ef-ad07-49c1-8ac5-b228340c6d7c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.343902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bb60f-23fc-4f4c-8582-af3bc066f6cd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.369697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.370141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.370424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.370698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.371429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:45.371741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:45.373806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:45.379926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 178/364] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:45.380683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.383467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.384410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.384845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.385337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.398936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:45.404777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:45.406023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:45.407047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:45.409455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 187/365] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:45.418207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.481455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.537593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.554709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce66b8-280e-4908-a9a4-ee8618fe2c0c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.593643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49324c-eb30-4ca5-9368-65b08fac90f6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.611020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.611637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.611910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.612176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.634833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.635487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.635753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.636023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.636685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:45.637076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:45.639236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:45.648552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 188/366] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:45.649567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.721406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:45.723765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:45.789274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb3d6-f822-4265-80e7-757701f50d98 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:45.790761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.829644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:45.836490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:45.837429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:45.837714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:45.837986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:45.845040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 179/367] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:45.845920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.858172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:45.867018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:45.867968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:45.868887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 189/368] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:45.870006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:45.971871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abd1d0-7f91-4f61-8f93-eded5bd29fd1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.021568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.021568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.021568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.021944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.028673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492aa8-b54d-4e3a-94c3-ee694b3ec422 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.061643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.076671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.078406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.078836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.079276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.081018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.083251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.085970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 190/369] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.087254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.117360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.119378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.176982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.235453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc1fd9-5754-4e1b-b65e-ea86e450566c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.251136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 180/370] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:46.252314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.287876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.288890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.289401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.289885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.290748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:46.291267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.292852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.305042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 191/371] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.306400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.343105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.403120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324539a-21f5-4161-950b-19c55a54a37d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.453409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.453950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f7316-8e00-446e-b493-1a9a45c55a4b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.454622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.455097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.455766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.458043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.459684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.461492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 181/372] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.462960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.494029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.511627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.512261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.512537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.512804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.514690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.517022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.522349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 192/373] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.522577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.605432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71697e2-f572-4d1e-9a63-10f865f55ad1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.642014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.652505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.653396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.653874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.654784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.664450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35961199-4046-455f-9dbf-3d510b6db7e1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.700234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.701538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.703165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.703165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.704041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:46.704445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.708980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.713274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 193/374] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.714178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.741864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.745690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.778782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:46.832830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd170cb-92cf-4df4-b38c-dcb8dfa4978d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:46.873102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 182/375] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:46.877351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.877685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:46.878531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:46.878984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:46.879433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:46.880797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:46.882459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:46.884332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 194/376] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:46.891269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:46.912792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:47.011015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e286f8-d3f7-43fb-858f-73b439b8b8a2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:47.018273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291f7bf-786e-49fb-9aa4-a28acd048bec None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:47.038911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:47.053317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:47.054242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.054690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.055143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.056300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:47.056793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:47.058571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:47.062442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:47.062957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.063238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.063533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] PATH_INFO: `/v3/projects/712d7cadf2c0478da279cd0f1a4dd259` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.064710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] RBAC: Authorizing `identity:get_project(project_id=712d7cadf2c0478da279cd0f1a4dd259)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:47.066220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:47.066585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 195/377] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:47.075038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 183/378] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:46 2022] GET /identity/v3/projects/712d7cadf2c0478da279cd0f1a4dd259 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:47.079932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:47.236175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d742-4265-468d-b05e-f360b7b07ac5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:47.283920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:47.284619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.284897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.285190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] PATH_INFO: `/v3/groups/3bb411f797f1428e8beee61e2f8227ce` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.290825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb411f797f1428e8beee61e2f8227ce)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:47.292299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e950f-addd-4c42-8485-8ff20539faa9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:47.295400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 196/379] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:47 2022] GET /identity/v3/groups/3bb411f797f1428e8beee61e2f8227ce => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:47.301893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:47.338555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2272d-1ed6-462d-a3fa-77f7459a3bc5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.338901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2272d-1ed6-462d-a3fa-77f7459a3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.339100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2272d-1ed6-462d-a3fa-77f7459a3bc5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.340422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 197/380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:47.347268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41990c-b71d-4a5c-a087-136c12cfcf6d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.347552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41990c-b71d-4a5c-a087-136c12cfcf6d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.347821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41990c-b71d-4a5c-a087-136c12cfcf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.378565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4e41990c-b71d-4a5c-a087-136c12cfcf6d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:47.380596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4e41990c-b71d-4a5c-a087-136c12cfcf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:47.459804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:47.491684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:47.504712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d904-e247-431c-8a9f-dca2d410fe94 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:47.511160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 198/381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:47.549730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:47.550817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.551295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.551760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.552749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] RBAC: Adding query filter params (group.id=3bb411f797f1428e8beee61e2f8227ce, role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=712d7cadf2c0478da279cd0f1a4dd259) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:47.553313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:47.555260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ac8e5-d55d-4e16-94cb-999b03d6e101 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:47.563186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 184/382] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:47 2022] GET /identity/v3/role_assignments?group.id=3bb411f797f1428e8beee61e2f8227ce&scope.project.id=712d7cadf2c0478da279cd0f1a4dd259&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:47.924539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2e74f-2baa-4f4f-8b0d-53f9b786b4ab None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.924968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2e74f-2baa-4f4f-8b0d-53f9b786b4ab None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.925175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2e74f-2baa-4f4f-8b0d-53f9b786b4ab None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.926697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 199/383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:47.932692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:47.933024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:47.933302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:47.956275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:47.957493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:48.051749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2e74f-2baa-4f4f-8b0d-53f9b786b4ab None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.109972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 185/384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:48.190907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2e74f-2baa-4f4f-8b0d-53f9b786b4ab None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.239184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.239808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.240081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:48.240347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:48.261832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:48.267302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:48.268275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:48.269215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 200/385] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:48.274399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.423084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:48.434226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380bb1e-f159-4736-b987-423ab9fb1935 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.480725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.481316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.481591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:48.481890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:48.482920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:48.483230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:48.484777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:48.488199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.495390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 186/386] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:48.501701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:48.616359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66dce7-44d0-49bb-8b2b-90b2a577da98 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.653366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.654444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.654937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:48.655427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:48.656871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:48.658620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:48.660772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 187/387] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:48.667319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.692597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64fb0d-4afa-4843-8ebe-6d4e5c086187 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.694557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:48.780489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a749627-a225-45d6-af18-e959b84e3ff3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.784905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.785648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.785982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:48.786287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:48.816790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:48.822146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:48.822316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:48.823141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:48.823921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 201/388] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:48.825786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.826274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:48.826749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:48.828160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:48.833943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:48.836139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 188/389] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:48.845941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.846115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:48.945350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:48.958840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd74d9-c30c-4474-91a5-c1ec093eb6f7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:48.998109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:48.999296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:48.999872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.000366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/groups/7da357e9817d445783f1fa0ca142b892/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.002800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, group_id=7da357e9817d445783f1fa0ca142b892, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:49.004443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:49.021387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 189/390] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 07:10:48 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/groups/7da357e9817d445783f1fa0ca142b892/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:49.036345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:49.040966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872c83-4d8d-4ba5-a363-d3d1f67d8f35 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:49.081206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:49.082191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.082673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.083145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.084069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:49.084595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:49.086245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:49.097397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 202/391] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:49.103624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:49.232788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883892-9ef2-49d0-b938-9064eb493e6b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:49.270629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:49.271999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.272548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.273338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.275003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:49.276624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:49.278776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 190/392] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:49 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:49.284627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:49.394500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141f1da-10f4-4df0-987f-37bac9bdf15d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:49.433921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:49.434930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.435923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.436391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.437968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:49.439717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612667e8-543b-48e7-b9a3-ff843bbaced8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:49.447434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:49.450705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 203/393] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:49 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:49.469969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:49.568625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ad6-b91a-4c99-85d5-cf54b2b26837 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:49.608395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:49.609560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.610059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.610529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/groups/7da357e9817d445783f1fa0ca142b892/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.612897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] RBAC: Authorizing `identity:create_grant(project_id=6c7c15b998df4b3cbaf8b1493173dbba, group_id=7da357e9817d445783f1fa0ca142b892, role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:49.614564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ddcad-6bf5-4c2c-8080-67ac44eed7ba None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:49.634113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 191/394] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 07:10:49 2022] PUT /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba/groups/7da357e9817d445783f1fa0ca142b892/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:49.869865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dca2-afef-4e07-bb54-c72c0ebab68a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.870214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dca2-afef-4e07-bb54-c72c0ebab68a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.870530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dca2-afef-4e07-bb54-c72c0ebab68a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.871977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 204/395] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:49.890766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.891111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.891316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.915091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:49.916649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:49.959076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d355-fc72-42be-8ada-2918bd76a6ff None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.959356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d355-fc72-42be-8ada-2918bd76a6ff None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.959527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d355-fc72-42be-8ada-2918bd76a6ff None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:49.961167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 205/396] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:49.966724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:49.967158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:49.967635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.013006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:50.013006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:50.016705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 192/397] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:50.040346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:50.088850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 206/398] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:50.335942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:50.345091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:50.449028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:50.470422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1b31-c453-4680-b3a3-6dd75df3a417 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:50.545713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:50.546355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.546626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.546892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.573506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:50.583246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:50.584150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:50.584930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:50.586335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 193/399] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:50.590978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:50.632967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cb2c0-09f4-4844-b013-0c4da82bec9f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:50.702483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:50.702584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.702862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.703158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.721233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:50.722995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:50.727915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:50.728803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:50.729621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:50.730226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 207/400] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:50.747901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:50.777690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:50.779333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc69999-94f0-4c3e-a3fa-e8cfd1a4d68e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:50.817223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:50.817864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.818243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.818520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.819296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:50.819605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:50.821001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571ff13-45aa-47f3-a425-e60a40cb4eca None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:50.830942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 194/401] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:50.832196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1991e8-cadf-4f42-b9bc-19339ead8ace None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.836993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1991e8-cadf-4f42-b9bc-19339ead8ace None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.837251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1991e8-cadf-4f42-b9bc-19339ead8ace None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.838947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 195/402] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:50.839703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1991e8-cadf-4f42-b9bc-19339ead8ace None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:50.866210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898851b-b804-459d-8717-fd0714a37c1c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:50.902591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:50.903295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.903557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.903970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.905079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:50.905258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:50.906611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceadc0-a3e8-40a4-aeb0-34f2efa3122f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:50.920350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 208/403] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:50.921806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:50.922093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:50.922367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:50.941640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:50.942769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:50.966589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1991e8-cadf-4f42-b9bc-19339ead8ace None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:50.999290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.008937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.009948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.010477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.010969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.012489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.014363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.016621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 196/405] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:50 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:51.016731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 209/405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:51.017645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.022448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.032072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.121682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d18a-b009-4150-bc22-c2d8eb4bda77 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.137378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9e2f-62ba-4734-9020-253376af983d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.157543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.158244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.158602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.158939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.160212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.161727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.163611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 210/406] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:51.168846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.176930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.177932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.178405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.178854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.180595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.182310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.184579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 197/407] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:51.194103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.300984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.323452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967966c-d4a6-4330-a21e-e8cf48115421 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.326817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.381424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.382159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.382431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.382697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.384668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.386132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.387925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 211/408] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:51.397447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.404407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87301dc-5a5c-453f-a883-fe667a92e70a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.458322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.459408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.459911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.460446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/groups/7da357e9817d445783f1fa0ca142b892/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.462695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, group_id=7da357e9817d445783f1fa0ca142b892, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.464448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.481392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 198/409] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 07:10:51 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/groups/7da357e9817d445783f1fa0ca142b892/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:51.482714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.523846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f282e-58ea-4572-bea1-59e400f7d018 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.562895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.563431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.563707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.563975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/groups/7da357e9817d445783f1fa0ca142b892/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.566594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a5c336ca884b43912266e3a18c8e0f, group_id=7da357e9817d445783f1fa0ca142b892, role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.568669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.585966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 212/410] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 07:10:51 2022] PUT /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f/groups/7da357e9817d445783f1fa0ca142b892/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:51.619225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9cfd-1cb0-4039-a584-9ac6ed7b64f7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.649619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.666011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.667033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.667525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.667997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.682255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:51.689253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:51.690340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:51.691130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:51.694031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 199/411] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:51.706114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:51.731230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:51.867237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60db2c4-2adf-4cd7-920d-1b54c54daa38 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:51.902774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:51.903305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:51.903577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:51.904475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:51.905172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:51.905716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:51.907094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0bb6-08a8-4f84-a443-f2bc3cb553b8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:51.916056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 213/412] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:51.922148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.064422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867f019-3d84-4e85-b51b-bb4a386da931 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:52.066646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe022448-c7dc-4eb7-aa89-d726c13bf810 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.066932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe022448-c7dc-4eb7-aa89-d726c13bf810 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.067202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe022448-c7dc-4eb7-aa89-d726c13bf810 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.068650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 214/413] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:52.074250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecccd-647f-42c8-8492-ccccee70bd5a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.074799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecccd-647f-42c8-8492-ccccee70bd5a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.075307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecccd-647f-42c8-8492-ccccee70bd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.109301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-371ecccd-647f-42c8-8492-ccccee70bd5a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:52.110498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-371ecccd-647f-42c8-8492-ccccee70bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:52.113706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:52.114345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.114623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.114896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] PATH_INFO: `/v3/groups/3bb411f797f1428e8beee61e2f8227ce` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.116075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] RBAC: Authorizing `identity:get_group(group_id=3bb411f797f1428e8beee61e2f8227ce)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:52.117599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:52.121271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 200/414] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:51 2022] GET /identity/v3/groups/3bb411f797f1428e8beee61e2f8227ce => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:52.125075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.227293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:52.285319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:52.286908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6a27d-75f5-441d-96db-6c3403cebadc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:52.295319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 215/415] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:52.314696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8963d-fd50-4c1f-b9a7-56631f16aeac None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.314991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8963d-fd50-4c1f-b9a7-56631f16aeac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.315247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8963d-fd50-4c1f-b9a7-56631f16aeac None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.316810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 216/416] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:52.322212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.322859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.323231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.333323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:52.333952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.334225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.334489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] PATH_INFO: `/v3/projects/712d7cadf2c0478da279cd0f1a4dd259` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.337986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] RBAC: Authorizing `identity:get_project(project_id=712d7cadf2c0478da279cd0f1a4dd259)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:52.340493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:52.343079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 201/417] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:52 2022] GET /identity/v3/projects/712d7cadf2c0478da279cd0f1a4dd259 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:52.349701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.359050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:52.360233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:52.489847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8634857-53ed-41b1-b292-7c53b901589a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:52.499157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 217/418] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:52.528437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:52.529254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.529531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.529806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] PATH_INFO: `/v3/projects/712d7cadf2c0478da279cd0f1a4dd259/groups/3bb411f797f1428e8beee61e2f8227ce/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.531998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] RBAC: Authorizing `identity:create_grant(project_id=712d7cadf2c0478da279cd0f1a4dd259, group_id=3bb411f797f1428e8beee61e2f8227ce, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:52.533526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:52.542013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:52.548161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 202/419] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 07:10:52 2022] PUT /identity/v3/projects/712d7cadf2c0478da279cd0f1a4dd259/groups/3bb411f797f1428e8beee61e2f8227ce/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:10:52.612477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.760122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef7e5-29ef-4e91-b1fb-338a27b330f9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:52.793008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:52.799324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:52.800026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:52.800305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:52.800571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:52.811942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:52.816826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:52.817716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:52.818486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 218/420] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:52.824128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.876720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:52.982199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a0316-fb46-42cc-8e19-3fcd06d28b08 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.042950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.043918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.045138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.046652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.047687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:53.049274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.051011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.065927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 203/421] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:53.068226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0e5d-4873-4aed-ad5c-f2f7eb1259f8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.070651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:53.132630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.134282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.134986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.135469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.150512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:53.156490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:53.164242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:53.165185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:53.165953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:53.175550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 219/422] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:10:53.180175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:53.235701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fb7f8-7089-4f4d-a398-870a04ff123d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.272926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.273604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.273889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.274147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.277750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:53.344248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.345682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.361516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaca68-6504-4c57-a980-e2526f062caa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.425178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.426704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.427490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.428220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.432611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:53.433173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.434737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.439955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:53.457426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 220/423] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:53.460300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:53.519969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 204/424] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:53.525500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:53.633135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753beb-bfd6-47ea-90b6-43fd8ed3b430 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.670655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.671731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.672197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.672679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.674224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.675978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.678281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 205/425] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:53.683477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:53.711879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:53.713966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618c9b-b7b3-4309-8982-46155b0f221c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.739077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:53.786075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.786685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.786959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.787224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.895194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240f96c-4441-430a-8d4c-317f250ab7f9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:53.896169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.898036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.966178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:53.966996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:53.967276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:53.967544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:53.969431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:53.970868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:53.972432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 206/426] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:53.978327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.065119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.079394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 221/427] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:10:54.085159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.089543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ffad1-194f-4e51-881d-7db083b4beaf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:54.109349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.127791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:54.128481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.128760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.129056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.129790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:54.130096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:54.131536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61143e75-356f-492a-930c-c537cd59b4fb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:54.139033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 207/428] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:53 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:54.235964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2186-8489-42b0-b854-4190c7276550 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:54.244767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ca3d-0e22-4da6-b7cf-9c940f84a565 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.245097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ca3d-0e22-4da6-b7cf-9c940f84a565 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.245343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ca3d-0e22-4da6-b7cf-9c940f84a565 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.246988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 208/429] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:54.254100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92a3d0-0012-4843-b752-1b052eb6cec4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.254371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92a3d0-0012-4843-b752-1b052eb6cec4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.254630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92a3d0-0012-4843-b752-1b052eb6cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.297239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8d92a3d0-0012-4843-b752-1b052eb6cec4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:54.298802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8d92a3d0-0012-4843-b752-1b052eb6cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:54.319795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:54.320505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.320851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.321150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] PATH_INFO: `/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.324856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] RBAC: Authorizing `identity:get_project(project_id=6c7c15b998df4b3cbaf8b1493173dbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:54.329319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:54.334229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 222/430] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/projects/6c7c15b998df4b3cbaf8b1493173dbba => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:54.339699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.415580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 209/431] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 28 07:10:54.424668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f17aee-d82a-4c5d-9633-3212be3051dc None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.425061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f17aee-d82a-4c5d-9633-3212be3051dc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.425255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f17aee-d82a-4c5d-9633-3212be3051dc None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.427538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 210/432] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:54.433386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.433662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.433927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.453945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:54.455110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:54.469433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.490919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fa975-eeff-44ef-a3e9-483ca68594ed None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:54.527517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:54.528305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.529226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.529500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.530648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:54.532075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:54.532586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 211/433] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:10:54.533932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 223/434] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:54.539684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.542574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.647186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c66f90-5b8a-4643-8060-8dacfb43443f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:54.683264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:54.683869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.684146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.684416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.686022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:54.686426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:54.687874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:54.699762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 212/435] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=6c7c15b998df4b3cbaf8b1493173dbba&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:54.726303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.874550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.883358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4a2c-7f1b-490b-b3c8-debe97f20b75 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:54.927546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:54.928487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:54.928763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:54.929068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:54.939945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:54.945482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:54.946422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:54.947194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 224/436] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:54.954410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:54.955973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.972310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:54.999753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fe26f-1f00-4a05-8cda-8e4673b42abb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.043931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.045386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.045698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.046521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.058621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:55.064433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:55.065406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:10:55.066221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 213/437] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:55.072355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.097145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c126-3162-4c1b-b589-8683993d36ec None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.151887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.152423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.152704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.153000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.154339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:55.154846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.156490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.166263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 225/438] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:10:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:55.173570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.220377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d4175-582f-4fc2-8c6e-e0408b60cf43 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.251191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.254293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.263804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.264515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.264816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.265119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.265791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:55.266094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.267457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c25126-af9f-4735-854b-e97ca1a161db None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.277093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 214/439] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:55.280151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4213bce-ffba-4194-bb46-ef340163ec12 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.280151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4213bce-ffba-4194-bb46-ef340163ec12 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.280151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4213bce-ffba-4194-bb46-ef340163ec12 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.281008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 215/440] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:55.285829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4213bce-ffba-4194-bb46-ef340163ec12 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.299056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035d131-4899-492d-92f0-fb498146b25b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.334366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.334963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.335238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.335498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.390138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4213bce-ffba-4194-bb46-ef340163ec12 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.401158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.402869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b1399-2f89-4cf0-9bf5-d0190c6a5026 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.423369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.423968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.424240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.424509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.469158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.478707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.491684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.493150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.501024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 226/441] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:10:55.502256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.502538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.502806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.521431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:55.522603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:10:55.588250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 227/442] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:10:55.589118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.599779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 216/443] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:55.601744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.691054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.705777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.709097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2027b-5219-40f3-bacd-b66f92226808 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.710525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d2d-6dc5-4ebd-b27d-a36d6816259c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.744407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.745069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.745328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.745595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.746682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.748506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.748619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.749147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.749421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.749711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] PATH_INFO: `/v3/projects/e4a5c336ca884b43912266e3a18c8e0f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.750818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a5c336ca884b43912266e3a18c8e0f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.752310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.754091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 217/444] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:10:55.754919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 228/445] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/projects/e4a5c336ca884b43912266e3a18c8e0f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:55.755629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.760274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.861757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542d6f4-9fd9-4921-9d06-56ae2ab0d968 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.867570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5ca4d-1ccd-4d38-aadd-dc2045b8f8d2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:55.895990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.896605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.896874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.897175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.898263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.899687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.901290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 229/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:55.901663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:55.902041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.902221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:55.902496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:55.902763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] PATH_INFO: `/v3/groups/7da357e9817d445783f1fa0ca142b892` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:55.904748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] RBAC: Authorizing `identity:get_group(group_id=7da357e9817d445783f1fa0ca142b892)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:55.906304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:55.907873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 218/447] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/groups/7da357e9817d445783f1fa0ca142b892 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:55.908631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:55.937877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:55.938359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.002545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818546-a748-4966-b8c1-3f94300ad517 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.012526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f8a2-9898-4a2d-94c6-b1ddedc7e3cc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.035618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.036240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.036513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.037024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.046628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:10:56.047418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.048008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.048284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.048554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.049301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=e036f4b567c14abab4c3011e4611ab47, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:56.049608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.051030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:10:56.051857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:10:56.052845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:10:56.053667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 230/448] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:10:56.054309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.062657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 219/449] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:55 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=e036f4b567c14abab4c3011e4611ab47 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:56.063872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.160967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.170967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ffc1-e89e-4680-ad6a-d47827136eb9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.173071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.187101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b9e64-2cdb-46b0-bf84-63052549a736 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.217641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.218429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.218702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.218969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.219711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] RBAC: Adding query filter params (group.id=7da357e9817d445783f1fa0ca142b892, role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=e4a5c336ca884b43912266e3a18c8e0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:56.220296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.221798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.225280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.225894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.226169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.226438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.227103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:56.227405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.233615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 231/450] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/role_assignments?group.id=7da357e9817d445783f1fa0ca142b892&scope.project.id=e4a5c336ca884b43912266e3a18c8e0f&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:56.233852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.246982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 220/451] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:56.253244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.369175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ae471-1399-4f3f-9248-64492b4c958d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.410737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.411313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.411591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.411864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.444090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.486638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.488113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.588550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 232/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:10:56.595242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.695517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7eda-77e6-4a63-ba4f-5db136003c9e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.727953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.728572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.728847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.729150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] PATH_INFO: `/v3/projects/712d7cadf2c0478da279cd0f1a4dd259` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.730261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] RBAC: Authorizing `identity:get_project(project_id=712d7cadf2c0478da279cd0f1a4dd259)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.732326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.733957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 221/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/projects/712d7cadf2c0478da279cd0f1a4dd259 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:56.739632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.796483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.835534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb10587-aa95-4261-b196-f99430dbab38 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:56.868657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:56.869235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:56.869512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:56.869784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] PATH_INFO: `/v3/groups/3bb411f797f1428e8beee61e2f8227ce` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:56.870868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb411f797f1428e8beee61e2f8227ce)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:56.872610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbae68-8f58-4625-99fa-1cd381350927 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:56.874180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 233/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/groups/3bb411f797f1428e8beee61e2f8227ce => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:56.879915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:10:56.899247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:10:56.977736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145e09-7284-4a28-8516-19c3af6e3473 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:10:57.009342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:10:57.009932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:57.010213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:57.010485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:57.011196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] RBAC: Adding query filter params (group.id=3bb411f797f1428e8beee61e2f8227ce, role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=712d7cadf2c0478da279cd0f1a4dd259) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:10:57.011504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:10:57.013476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de72572-61fa-4f06-9976-0b05eaa0d5b3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:10:57.019175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 222/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 07:10:56 2022] GET /identity/v3/role_assignments?group.id=3bb411f797f1428e8beee61e2f8227ce&scope.project.id=712d7cadf2c0478da279cd0f1a4dd259&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:10:59.931444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a2d7-5246-4c1c-b43f-b8fa2177ebc9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:59.932480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a2d7-5246-4c1c-b43f-b8fa2177ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:59.932985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a2d7-5246-4c1c-b43f-b8fa2177ebc9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:59.934758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 234/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:10:59.940576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa4324-6e65-4ed0-be55-a2836dbdf20a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:59.941414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa4324-6e65-4ed0-be55-a2836dbdf20a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:59.941895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa4324-6e65-4ed0-be55-a2836dbdf20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:59.959042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129a59-9d6b-4feb-a618-778bf62429be None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:59.959624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129a59-9d6b-4feb-a618-778bf62429be None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:59.960088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129a59-9d6b-4feb-a618-778bf62429be None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:59.962006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 235/457] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:10:59.972980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863f4c9-d360-43d1-8808-69173b539fc3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:10:59.972980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863f4c9-d360-43d1-8808-69173b539fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:10:59.972980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863f4c9-d360-43d1-8808-69173b539fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:10:59.974758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4afa4324-6e65-4ed0-be55-a2836dbdf20a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:10:59.976226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4afa4324-6e65-4ed0-be55-a2836dbdf20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:00.002130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1863f4c9-d360-43d1-8808-69173b539fc3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:00.003358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1863f4c9-d360-43d1-8808-69173b539fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:00.070630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 223/458] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:00.090538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 236/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:00.167688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae6f66-04f6-4208-8646-d6c0c68f92af None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.168365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae6f66-04f6-4208-8646-d6c0c68f92af None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.168828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae6f66-04f6-4208-8646-d6c0c68f92af None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.170540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 224/460] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:00.176747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.177383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.177860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.196782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:00.198309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:00.274087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.275757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715e3a-2a83-4906-951d-d2f5ba5f3d48 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.276037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715e3a-2a83-4906-951d-d2f5ba5f3d48 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.276425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715e3a-2a83-4906-951d-d2f5ba5f3d48 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.279646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 225/461] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:00.283699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.283976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.284244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.301532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 237/462] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:00.309973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:00.311171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:00.318960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.377217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.425188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 226/463] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:00.429742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.566293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591b952-8ac3-420a-9563-c6b4b8146bf0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.608837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ee4f6-e8c3-4fad-a21a-5a0c555ab93c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.621203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:00.622284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.622728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.623454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.637160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:00.640262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:00.647709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:00.648575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.650143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.650613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:00.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:00.661661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:00.662482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:00.662482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.662482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:00.662482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 238/464] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:00.663123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.671216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:00.672905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:00.692636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:00.693606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:00.694456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:00.694456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.694456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:00.695452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 227/465] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:00.698565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.698880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.722943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.784632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bdcc-b03e-4a16-a2b8-fa94f06d045f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.808816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c973-977a-4a0f-b715-328e38b71056 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.820558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:00.821144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.821418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.821685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.832910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:00.834392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:00.845228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:00.845831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.846107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.846374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.847011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:00.847313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:00.848655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:00.849262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:00.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:00.850892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:00.850892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:00.850892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:00.850892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 239/466] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:00.851110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.870743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 228/467] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:00.870743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:00.930855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.949703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:00.959686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44066b-f960-4c6c-8099-da0b4a728319 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.981237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9385b6-5859-40d6-9afa-edf26039ae7b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:00.994811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:00.995417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:00.995698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:00.995972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:00.996624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:00.996933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:00.998865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.016700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 240/468] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:01.017564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.021896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.022720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.023173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.023631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.034690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.036328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:01.053035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:01.053949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:01.054776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:01.054776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:01.054776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:01.056414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 229/469] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:01.057333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.131778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e2bb-6042-454b-9b9e-0ffafbe7883f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.162924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dc00-981b-4982-a10f-f6ce2a3165a2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.168701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.169440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.169714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.170059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.170831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:01.170960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.171380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:01.172345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.185133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 241/470] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:01.185796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.201875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:01.207699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.208305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.208583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.208856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.209541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.210971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.244983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 230/471] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:01 2022] POST /identity/v3/users => generated 248 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:01.244983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.332692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463141c2-eab6-45f9-98c7-52655712b6dc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.371625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177192-1b4d-47b3-963c-45bae0a62fff None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.376099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.376717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.377012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.377282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.377952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.379428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.402658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 242/472] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:01 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:01.403872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.425203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.426190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.426758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.427135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.427909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:01.428362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.430131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc6c-dd9a-43ee-b5dc-9b03326a47fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.447837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 231/473] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:01.449883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc51fc-2f99-4812-a4b1-02e0a84d3f72 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.449883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc51fc-2f99-4812-a4b1-02e0a84d3f72 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc51fc-2f99-4812-a4b1-02e0a84d3f72 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.456485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 232/474] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:01.456485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc51fc-2f99-4812-a4b1-02e0a84d3f72 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:01.484742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:01.502968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:01.549830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b873b81-31db-4105-88ac-fcdf3928cbcf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.592544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.593285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.593525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.593799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.594473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.595906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43482f9-f5b8-4a11-be07-db95f9c0ff96 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.596018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc51fc-2f99-4812-a4b1-02e0a84d3f72 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:01.624412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 243/475] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:01 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:01.625673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.625953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.626219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.645562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:01.646772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:01.647850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:01.648479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:01.648868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:01.649288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:01.650067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:01.651522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:01.693434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 233/476] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:01 2022] POST /identity/v3/users => generated 249 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:01.746708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 244/477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:02.033354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:02.117280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:02.149750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b1bb-3a80-4867-bf45-d809b02057c2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:02.202416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:02.203365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:02.203887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:02.204348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:02.205295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:02.206878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:02.214706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.sql.core [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 07:11:02.214706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 07:11:02.214706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [parameters: {'id': '595bb2b8cbcd4006a2d884bcd11125de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 07:11:02.214706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result.read() Sep 28 07:11:02.232366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return ref.to_dict() Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:11:02.237595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application yield resource Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise self.value Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application session.commit() Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 07:11:02.238478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.session.flush() Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application compat.raise_( Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 07:11:02.239315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:11:02.240137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result.read() Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:11:02.242062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [parameters: {'id': '595bb2b8cbcd4006a2d884bcd11125de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:02.243977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 07:11:02.245927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:02.253303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 234/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 07:11:02 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:02.255569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:02.359658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:02.430739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68133-ecc7-4e90-acf0-09f6e389e8a8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:02.471416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:02.472057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:02.472336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:02.472607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:02.488836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7abd7c06-b298-4774-9146-b079b73716f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:02.493688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:02.495762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:02.498237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 245/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:02.505589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:02.616572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef26a5-1011-4149-b434-752d4012e1fc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:02.660519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:02.661172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:02.661448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:02.661715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:02.662390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:02.662687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:02.664045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919baa94-ee15-4cb7-8a77-61927f1d9f02 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:02.673599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 235/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:03.871436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433d7cd-d4e6-43d9-9c3e-85b07cadef63 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:03.871436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433d7cd-d4e6-43d9-9c3e-85b07cadef63 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:03.871436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433d7cd-d4e6-43d9-9c3e-85b07cadef63 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:03.871436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 246/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:03.878724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81473-9d0e-4260-8e02-27b29b945c63 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:03.879296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81473-9d0e-4260-8e02-27b29b945c63 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:03.879296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81473-9d0e-4260-8e02-27b29b945c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:03.889760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:03.905987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-27d81473-9d0e-4260-8e02-27b29b945c63 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:03.907170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-27d81473-9d0e-4260-8e02-27b29b945c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:04.000538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 236/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:04.301988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95f7-5927-41db-b280-721747049f72 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.302285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95f7-5927-41db-b280-721747049f72 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.302555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95f7-5927-41db-b280-721747049f72 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.304101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 247/483] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:04.311707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.311991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.312261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.335876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:04.337079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:04.403142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79143-158e-4d1e-acdc-0654471bdd4d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.404143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79143-158e-4d1e-acdc-0654471bdd4d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.404420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79143-158e-4d1e-acdc-0654471bdd4d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.406499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 248/484] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:04.412904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c8bb-1743-4f58-8104-cfece51bfef1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.413348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c8bb-1743-4f58-8104-cfece51bfef1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.413623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c8bb-1743-4f58-8104-cfece51bfef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.432023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 237/485] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:04.433368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:04.448826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e5d8c8bb-1743-4f58-8104-cfece51bfef1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:04.450906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e5d8c8bb-1743-4f58-8104-cfece51bfef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:04.550527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:04.576144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 249/486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:04.577725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13f2b8-a3c4-47f5-b8e3-88d81d68a659 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.578008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13f2b8-a3c4-47f5-b8e3-88d81d68a659 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.578665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13f2b8-a3c4-47f5-b8e3-88d81d68a659 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.584570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 250/487] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:04.586113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.586396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.586665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.597703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63dbb0-9bec-4b81-ad1c-dfb5d25f2875 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:04.610825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:04.612257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:04.658410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:04.680211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:04.681133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.681421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.681679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.691762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 251/488] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:04.713846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:04.719495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:04.720525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:04.721366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 238/489] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:04.726778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:04.771975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:04.898132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ceef5-f5be-4366-ae13-6d4fb0ab5252 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:04.899667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546ac8-a7f1-4210-87c9-c0b2dc8eb013 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:04.924329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:04.937335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:04.938138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.938555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.938967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.940292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:04.940987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:04.941200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:04.941466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:04.942134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:04.942438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:04.943813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:04.949953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:04.952911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 252/490] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:04.953782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:04.955398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:04.956283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:04.957114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 239/491] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:04.965904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:04.997958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.074497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de797b-3039-46f9-9cee-0e7f012b8fb8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.079324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241e6c-7f84-4c6e-a720-bd00a145c424 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.108044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.108671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.108968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.109291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.116223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.117146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.117569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.118004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.119149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:05.125017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.126086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 253/492] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.131843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.133891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.135355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:05.151653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.152562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:05.153651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:05.153651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.153651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:05.153651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 240/493] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.157980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.202588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.236535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.243858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ccc2a-3f55-4b11-98a2-be7c68ebc64b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.277736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da58ab-7965-441f-b699-6c29bc0c2546 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.280925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.281585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.281867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.282135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.282791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:05.283091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.284457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.295246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 254/494] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:05.295924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.320148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.320768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.321094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.321361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.331972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:05.337176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.338055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:05.338844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 241/495] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.339484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.406066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d058a-228b-4eee-a4a9-3011f9d792e0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.441538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.441730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.442792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.443162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.443506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.444295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:05.444642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.446027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.448700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5fa54-a39f-4112-8b1a-1106ffb15a7e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.454059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 255/496] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:05.454865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.478934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.487069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.487665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.487938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.488244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.488834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:05.489159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.490518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.503056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 242/497] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:05.508598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.566170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c2cd-f95d-4e4d-b8dd-0be1983da065 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.601684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.602317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.602597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.602866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.614260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.615703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.616387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53967-2bdb-498f-9b83-6025cd7fd9e8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:05.630342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.631446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:05.632448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:05.632448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.632448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:05.636287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 256/498] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.637212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.654739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.655920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.656469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.656932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.657827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:05.658243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.659684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.669640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 243/499] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:05.670533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.690358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.714988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.746596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2ec7-af7e-4567-837d-ccff39bed7d2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.773840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76bf08-5e6a-4358-847e-ed39c3d987fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.778753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.779419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.779691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.779956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.790088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.791526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:05.805940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.806858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:05.807699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:05.807699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:05.807699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:05.807699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 257/500] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.807920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.808003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.808070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.808144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.808400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.818830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.820248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d079dd51-2da0-44d3-ac3d-76e64162d7a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:05.832577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:05.833700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 244/501] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:05.835002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983fb2-171c-476f-8d50-65338201d812 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.835673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983fb2-171c-476f-8d50-65338201d812 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.835673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983fb2-171c-476f-8d50-65338201d812 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.836502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 245/502] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:05.837212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4983fb2-171c-476f-8d50-65338201d812 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.915250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.915870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd4261-b28b-4f6c-a89a-5849d6fa2bc7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.938509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:05.939197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4983fb2-171c-476f-8d50-65338201d812 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:05.949193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.949789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.950063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.950919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.951562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:05.951861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.953243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:05.964493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 258/503] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:05.965311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:05.975398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:05.976018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:05.976294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:05.976564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:05.987342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:05.988775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:06.003819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:06.004899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:06.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:06.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:06.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:06.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 246/504] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:06.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.065508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cd243-6095-46bf-b4db-8309dcf25686 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.099587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.100183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.100459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.100723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.101391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.101686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.103047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894537c-993f-4d4b-8158-f45d6dc3c446 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.107501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcc2fa-2bad-4827-ab4f-e00b2fd467d1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.114200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 259/505] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:06.115357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.115635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.115904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.133022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:06.134179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:06.138317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.144727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.145353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.145627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.145895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.146616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.146917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.148320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.174283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.185495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 247/506] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:06.186355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.201120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 260/507] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:06.202031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.301171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4294-b186-416e-a8ef-80febaf8c390 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.309029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3708a0-113b-47ed-8a48-a8ebfa812d2d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.338870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.339489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.339767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.340034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.345290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.345893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.346165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.346432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.347066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.347361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.347797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.348742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.349589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.360230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 261/508] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:06.361312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:06.362966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:06.363833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:06.364851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 248/509] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:06.365505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.403151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.420017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.470810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5bfa-b419-44ab-901f-ad69f0e2e938 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.477892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234c1d0-2cf9-4240-a0c5-97db26de7a65 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.504000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.504606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.504876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.505167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.510789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.511382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.511662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.511930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.512292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.514548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:06.526676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:06.527538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:06.528690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 262/510] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:06.529295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.585238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.586696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.620590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.627802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929ae6-bbdb-462c-8c35-9e1186231cfa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.634013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.661361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.662026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.662297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.662560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.670298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.671752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:06.682713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:06.683617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:06.684507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 263/511] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:06.685649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.686025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 249/512] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:06.686759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.787068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472dba-1a3a-4346-896b-ea887bfbc04a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.787252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8613b-07ab-40d4-9219-ec70c1dc852f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.819351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.819956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.820235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.820505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.820800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.821278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.821511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.821596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.821784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.822050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.822996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.832434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.833879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.842549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:06.849259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:06.850321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:06.851338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:06.851338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:06.851338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:06.851924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 250/513] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:06.852885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.861746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 264/514] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:06.862596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:06.876677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:06.954148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e871a2-3c19-4a39-912a-9e5dcc1ad77c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.960293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa8a6-c120-4a15-b463-3864478ca056 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:06.987221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.987813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.988088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.988354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.989103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.989405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.990813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:06.993377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:06.993972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:06.994250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:06.994522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:06.995248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:06.995551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:06.997508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.007759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 251/515] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:07.008552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.016344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 265/516] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:07.017189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.069103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.086790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.110571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95975-d889-4dbf-bd5d-95941e3f9257 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.120245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b819-65cd-4318-bc07-ef92912cbc5b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.147687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.148223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.148493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.148760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.154323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.154911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.155187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.155449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.156070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:07.156371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.157751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.158765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.160179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.168735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 266/517] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:07.169638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:07.176675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:07.177647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:07.178469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:07.178469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:07.178469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:07.178469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 252/518] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:07.178959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.271392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452701f7-db59-4ef9-a4db-8f2639dc27d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.278258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340a6f-e937-43f3-96d8-d65e1e9cd679 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.288408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.300391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.306564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.307160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.307437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.307712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.313054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.313647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.313924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.314191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.374262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.375647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.378983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.380409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.472050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 267/519] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:07.472895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.482267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 253/520] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:07.482939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.495440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.501793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.571105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871106c5-e239-42fb-a086-9f89b1686b82 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.581520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bad3-d066-4ba1-a998-e322369cff0f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.604202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.604795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.605098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.605368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.614596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.615188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.615459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.615726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.616360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.617808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.638725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 254/521] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:07 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:07.639466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.684022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.685655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.722186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.724623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.764463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4823e-ba9f-40c4-a117-6c09df3093d4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.804488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.805137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.805661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.805927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.806550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 268/522] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:07.806649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:07.806863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.807403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.808309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.823633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 255/523] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:07.824365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:07.923396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a70a-91cb-479d-95b9-ae270ed327a8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.935341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840a49-3831-4212-a40d-ca9186856c78 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:07.955629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.962717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:07.963536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.964167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.964520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.964738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.972469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:07.973128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:07.973401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:07.973664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:07.976289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.977796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.984044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:07.985500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:07.993179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:07.994088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:07.994985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:07.994985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:07.994985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:07.999235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 269/524] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:07.999235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:08.005218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.006128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:08.007066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:08.007066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:08.007066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:08.007066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 256/525] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.007297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.111371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c4ba0-db40-4427-b794-2fdabc9ab911 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.112717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77b41f-ca43-4ac4-95e6-a32924c7ccd9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.147276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.147902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.148273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.148547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.148767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.149426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.149700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.149964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.158658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.160107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.162649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.164273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:08.172086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 257/526] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.174430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:08.180655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.181702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:08.182543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:08.182543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:08.182543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:08.185245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 270/527] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.186422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.220552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.225489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.285048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f9aa-de6c-4c6d-b401-5d7d907d39ec None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.297545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ab08-3a70-4449-b3fb-ca95a9ae8f83 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.319598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.320446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.320909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.321211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.322095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:08.322404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.324127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.333701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.334298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.334667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.334903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.335457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:08.335756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.341890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 258/528] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:08.342024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.342114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.351937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 271/529] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:08.352766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.447989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.452765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc8f0b-5eef-4686-a0ab-cac72f2e3ce2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.456118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.464583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bb1d8-b201-4d33-9f46-de9c77b2d0cd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.489930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.490527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.490798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.491310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.491739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:08.492038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.493547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.499562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.500150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.500425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.500690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.501341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:08.501638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.502994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.516662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 259/530] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:08.516662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.520702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 272/531] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:08.525071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.624074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4064-4356-4321-a672-7da3db191734 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.628160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f7c6-ac09-464a-8680-3352b3ce312a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.658925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.659543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.659815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.660115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.662460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.663072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.663346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.663610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.667088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.668520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.670237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.671655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:08.679760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.680848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:08.681723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 260/532] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.682181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:08.687149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.688142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.689178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:08.697478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 273/533] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.697478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.706721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.715319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.791006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d395-e4f1-4fa7-9753-7807eafb7fa3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.801042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2fba0-099e-4942-afd0-9d2b29f6e864 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:08.827625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.828293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.828565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.828832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.829645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=20ca1a72134341ed9dbe8b58c339873e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:08.829950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.831604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.836027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:08.836652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:08.845403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:08.845613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 261/534] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/role_assignments?user.id=20ca1a72134341ed9dbe8b58c339873e&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:08.845696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:08.845778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.853319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:08.855460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:08.871442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:08.872869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:08.873867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:08.877143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 274/535] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:08.878212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:08.964406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.981817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:08.991105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933d62-7d5d-4084-9b6b-e9412550a28a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.014725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f44991-b025-4b92-a788-af645dc871ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.034731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.035341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.035611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.035879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.036576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.036880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.038357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.052602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 262/536] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:09.053557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:09.058448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.059262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.060329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.060740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.061721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.062164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.063877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.083662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 275/537] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:09.084805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:09.167412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0965a-76f8-46f1-916c-13d1244ee2c6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.195368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791651-b5ef-48d5-8515-f15e8dbf7b7f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.209758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.210609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.210976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.211789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.212279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.212674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.214208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.229827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 263/538] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:09.230873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:09.234086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:09.235388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.236160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.236594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.237288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.238402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=170658d3fbf547a09a4dd2bb77ecb01a) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.238914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.240552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.243874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:09.250360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 276/539] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:09 2022] GET /identity/v3/role_assignments?user.id=170658d3fbf547a09a4dd2bb77ecb01a&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:09.251575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:09.361167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ca72-6254-4ae0-b7f7-a7f7e46a8b45 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.396078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bf41-a982-42a8-b824-e42173fd99e5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:09.423812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.424598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.424985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.425353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.427174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.427620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.429370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d740-edc3-484b-9200-13dba9d31891 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.451958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 264/540] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:09 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:09.460212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:09.461013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:09.461389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:09.461752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:09.462663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=ce964ecd12bd4e3d9d85bf5dc227d35f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:09.463087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:09.464661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e059d-76b0-42f0-80b5-437d08f8b89b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:09.471694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 277/541] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:09 2022] GET /identity/v3/role_assignments?user.id=ce964ecd12bd4e3d9d85bf5dc227d35f&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:10.019276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee549cd-17cf-4d01-878e-1959be7fffad None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:10.019777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee549cd-17cf-4d01-878e-1959be7fffad None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:10.019777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee549cd-17cf-4d01-878e-1959be7fffad None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:10.021357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 265/542] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:10.026817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:10.027226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:10.027412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:10.047078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:10.048278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:10.144971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:10.160986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 278/543] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:10.526100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee549cd-17cf-4d01-878e-1959be7fffad None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:10.622738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:10.671377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee549cd-17cf-4d01-878e-1959be7fffad None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:10.728373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:10.730042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:10.730042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:10.730042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:10.740885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:10.746493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:10.747958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:10.748717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:10.750853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 266/544] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:10.756630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:10.886736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b17ed6-8fa3-432d-98a6-ce08c23f55d6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:10.921163 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:10.921763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:10.922037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:10.922305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:10.923028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:10.923334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:10.924698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:10.933334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 279/545] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:10.939468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.104724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51194d-048e-48b0-9f5d-f51bd6199b15 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:11.144675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:11.161878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:11.162556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.162837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.163112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.181515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:11.182993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ab4bf6f9-2014-4fe3-b65a-154c647b063a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:11.201674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:11.202593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:11.203408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:11.203408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:11.203408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:11.204921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 267/546] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:11.209664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.275263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:11.301505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42fe6-62c7-4d83-b586-d408607c3e15 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.301796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42fe6-62c7-4d83-b586-d408607c3e15 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.302049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42fe6-62c7-4d83-b586-d408607c3e15 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.303381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 268/547] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:11.308546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.308831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.309136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.342295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168afe6-df49-46fa-b3f2-ff1c530e6f92 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:11.346284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:11.347476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:11.404456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:11.404996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.405275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.405545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.406188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:11.406490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:11.407852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:11.431292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 280/548] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:11.437318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.461512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 269/549] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:11.587805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3614a73-ccf3-49fb-9265-6b61ff9234c7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:11.603855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:11.635512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:11.636142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.636420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.636691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.648230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:11.649705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:11.662735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:11.664217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:11.665602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 281/550] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:11.669997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.777434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.807798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c652b8-7350-444c-ac74-f2bf811ff27b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:11.823241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:11.866932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:11.867652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.867932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.868206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.868939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:11.869325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:11.871073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:11.886635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9ec4b-e6aa-42e6-894b-0766d1de15fd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:11.908565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 270/551] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:11.914467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:11.928079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:11.928715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:11.929023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:11.929273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:11.944177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:11.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:11.952227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:11.953056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:11.958528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 282/552] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:11.964277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.013406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.119932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9275a7-384f-4690-ba8c-bfb6349c63b6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.124004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e5eb-bf70-4a8f-89b7-6fe95b9bd454 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.164185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.164858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.165109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.165389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.191369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.193543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.193824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.194099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.194771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:12.195078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.196745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.218589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 283/553] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:12.225933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.252164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.321987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.323336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fcbe0-10a5-43dd-b125-97e8fd6ed952 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.369360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.377084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c29b2-05cf-4699-a280-94c420d9009e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.417948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.419425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.419832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.420249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.431389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.432975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.440328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 271/554] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:11:12.442644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481b00-e36f-4af0-8669-eca3976cbce9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.443101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481b00-e36f-4af0-8669-eca3976cbce9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.443533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481b00-e36f-4af0-8669-eca3976cbce9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.444976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 272/555] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:12.446154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a9e2-c53b-4ec1-b2d2-16dd05b0df91 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.446596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a9e2-c53b-4ec1-b2d2-16dd05b0df91 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.447027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a9e2-c53b-4ec1-b2d2-16dd05b0df91 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7212f12c-c59b-4433-8d80-40b29b7b6820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:12.447177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:12.448032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:12.448879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:12.448879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:12.448879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:12.450705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 273/556] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:12.451607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06a9e2-c53b-4ec1-b2d2-16dd05b0df91 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.453235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 284/557] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:12.454502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.454793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.455062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.474465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:12.476366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:12.528683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.550022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 285/558] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:12.550906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.577023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06a9e2-c53b-4ec1-b2d2-16dd05b0df91 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.616101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.617144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.617420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.617692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.628959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.630431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.634670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-13badd33-9c3d-48f0-8a7a-0bcf6fb63252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:12.646843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:12.647768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:12.648666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:12.648666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:12.648666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:12.648666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 274/559] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:12.649331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.649331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.649616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.666185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daba5c-4ab8-4bab-89bd-e203ff850454 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.668192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:12.670338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:12.708790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.709453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.709725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.709988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.710627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:12.710926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.712301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2716397-ec56-4103-8c43-7e468124f01f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.724663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 286/560] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:12.730045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ee18d-777b-434d-a20c-179adcc88e11 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.730329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ee18d-777b-434d-a20c-179adcc88e11 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.730601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ee18d-777b-434d-a20c-179adcc88e11 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.731980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 287/561] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:12.732667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ee18d-777b-434d-a20c-179adcc88e11 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.747564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 275/562] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:12.748400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.801268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.860118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923a4a-238c-4b61-850a-da43a4f032cc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.877238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ee18d-777b-434d-a20c-179adcc88e11 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:12.897838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.898455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.898728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.898991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] PATH_INFO: `/v3/users/glance` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.907484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.908928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.914400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:12.915065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.915366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.915656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.916341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:12.916645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:12.918077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-65c89834-bbda-4b69-8bfb-923667956abe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:12.922247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:12.923343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:12.924464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:12.925377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:12.929099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 276/563] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:12.930363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:12.930911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:12.931154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:12.935223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 288/564] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:12.936508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:12.953116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:12.954323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:13.023269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 277/565] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:13.024377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.054247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec9e-4800-42f5-b52c-103fde105f2e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.067663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.090374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.091235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.091521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.091778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.101984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:13.107432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:13.109266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 289/566] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.109950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.138427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e15441-d1e6-490d-a5df-3299d1677e20 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.174677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.175619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.176251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.176506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.177166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.178025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:13.178460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.179881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.204220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 278/567] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:13.204220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.233059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec483e-d6b3-46ed-98dc-873b0f5ece9a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.270068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.270988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.271474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.271892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.280270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.282564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:13.295410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.296375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:13.297291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 290/568] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.297752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.313265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.319652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73132a-0da6-43c2-b145-0448296734e0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.354795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.355420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.355702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.355998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.366222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:13.371217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.372714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:13.373744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 279/569] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.375083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.400402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d1be-2bc6-4ee4-87ab-2e4a389fef86 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.416801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.435549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.436052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.436326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.436590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.437269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:13.437564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.438919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.446439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 291/570] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:13.451383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.474699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4345-e674-40b5-b2d6-aa519f345cd1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.508139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da774faf-1739-4b31-a031-1282e446507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.508755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da774faf-1739-4b31-a031-1282e446507f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.509049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da774faf-1739-4b31-a031-1282e446507f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.509315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da774faf-1739-4b31-a031-1282e446507f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.519464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da774faf-1739-4b31-a031-1282e446507f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.520912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da774faf-1739-4b31-a031-1282e446507f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-da774faf-1739-4b31-a031-1282e446507f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:13.535009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.536056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:13.536939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:13.536939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:13.536939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:13.536939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 280/571] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.536939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da774faf-1739-4b31-a031-1282e446507f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.553512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.560973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353dce-d71a-46c2-8678-d8a436a66221 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.593891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.594742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.595012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.595278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.604973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:13.610670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.611994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:13.613229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 292/572] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.615004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.646555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da774faf-1739-4b31-a031-1282e446507f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.656108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.681287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.681885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.682155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.682419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.683100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:13.683401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.685401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.698563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 281/573] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:13.698713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.722842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504d518-a584-4bcc-af0b-6af6ef922f00 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.757863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.758447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.758717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.758983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.759832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:13.759941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.761286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.766940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.773708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 293/574] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:13.773708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.800811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de393a-20de-4e92-a1ad-99e59f1e3ae6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.834265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.834883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.835160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.835430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.845600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.847029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:13.864755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:13.865943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:13.866916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:13.866916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:13.866916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:13.866916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 282/575] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:13.867918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.868070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951479-6473-40b2-ba14-b0085ac8a81b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.869834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:13.907170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:13.907874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:13.908214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:13.908484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:13.909145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:13.909442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:13.910785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:13.922721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 294/576] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:13.923507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:13.972580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823089c9-75fc-4121-ba1a-4596ef0c86a8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:13.991760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.006946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.007956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.008233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.008652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.009336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.009640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.011008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.019336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8edf-f1dd-4d3a-9e91-37bdc6cf6bef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.019739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 283/577] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.020529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.054036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.054624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.054895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.055165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.056148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=c4389bd42da64dccbf958503dccf87e1) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.056453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.057886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.063048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 295/578] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:13 2022] GET /identity/v3/role_assignments?user.id=c4389bd42da64dccbf958503dccf87e1&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.063824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.100023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.129835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcadbb1-4c65-4b3d-9956-54f0dda5e66c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.168871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.169716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.169982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.170250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.182411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.182941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8676-c497-474b-b6a1-3b891dfe7814 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.183883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:14.201452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:14.202543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:14.203493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:14.203493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:14.203493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:14.203493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 284/579] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:14.203493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.228064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.228584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.229353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.229637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.230339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.230580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.232014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.244774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 296/580] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.245515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.246022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.330058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fcf3-e114-4ce0-9f3e-5fd1f9d8fd29 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.336369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.353157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9137589-8bb5-42dd-ab43-b5fbfca2f043 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.366598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.367306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.367602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.368176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.375607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.377068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:14.389788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:14.390708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:14.391724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:14.392176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.392176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.392176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.392176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.395563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 285/581] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:14.396401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.403210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.405236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:14.420644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:14.421633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:14.422518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:14.422518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:14.422518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:14.422518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 297/582] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:14.422518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.502186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.510700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2768c03-d6e1-4495-81e0-2be28277bcaf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.530388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c956a-2a5d-4019-96e7-7ead20bd4f3c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.549082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.549689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.549963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.550230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.550863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.551170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.552522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.566022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.566553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 286/583] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.567281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.571187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.571476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.571754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.580751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.582214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.584181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:14.597968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:14.599142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:14.600300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:14.601155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 298/584] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:14.601963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.677199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bbfc5-ab38-4f79-9c1f-102f1bcb01c0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.709597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8de32d-5b28-4e27-b571-656c8a1d5bfb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.718992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.719603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.719879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.720147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.720838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.721164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.722576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.736052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 287/585] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.736777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.750511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.751118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.751387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.751658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.752349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.752656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.753990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.754103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.768612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 299/586] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.769504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.831660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:14.852687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1597c-a6c4-4b61-82ec-1cae4219995a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.876124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43062394-6e48-47aa-82b2-232a28750ed4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:14.889239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.889999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.890375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.890747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.898442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.899981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:14.911534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:14.913356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:14.915197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:14.916221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 288/587] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:14.916911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:14.924826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:14.925484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:14.925762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:14.926030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:14.926779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:14.927087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:14.929393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:14.951940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 300/588] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:14.953398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.006655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.035982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9fe6-07a6-4bb9-8d79-5b1aa4203c11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.062413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045c0fe-6517-4626-afd9-47912995888a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.071760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.072746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.073068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.073345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/20ca1a72134341ed9dbe8b58c339873e/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.078502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.090985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=20ca1a72134341ed9dbe8b58c339873e, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.092524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.096517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.097173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.097463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.097736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.104878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.105400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 289/589] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:14 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/20ca1a72134341ed9dbe8b58c339873e/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:15.106860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.107006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:15.130810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:15.132134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 301/590] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:15.133256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.258007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489b66-5322-49e9-91fd-26cef119135f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.292219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2b27-ee33-4889-9723-db2547e0f9e3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.296441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.314376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.315430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.316229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.316669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.317588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:15.318054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.320471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.339886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.340222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 290/591] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:15.341054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.341304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.341571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.342048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.358060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.359959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:15.376186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:15.377420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:15.378316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:15.378316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.378316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:15.380377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 302/592] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:15.381441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.412045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.458845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e104dd-7a61-4c57-b819-79a65d892e81 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.498365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.499347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.499680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.500015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.500798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:15.501188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.502661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.507322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759d100-48f7-4759-b82e-83e53bbb4702 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.521374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 291/593] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:15.522227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.552794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.554438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.555178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.555896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.566835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.573268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.575487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:15.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:15.606524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:15.607472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:15.607472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.607472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:15.607472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 303/594] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:15.607811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.649019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d387-c1dd-4ecc-b872-9b94d7abb751 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.696601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.698124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.698548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.698972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.699808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:15.700828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.702562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.718887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 292/595] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:15.720236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.722257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.744775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392931ce-c8f8-4ec2-8258-ee37043845f9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.784018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.784739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.784961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.785238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.796732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.798221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:15.813591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:15.814768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:15.815733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:15.815733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:15.815733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:15.819445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 304/596] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:15.820324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.852904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.859432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49e819-9fe9-4876-b6de-a9e57bb48122 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.908836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.910213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.910635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.911464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.912542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:15.913018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.915284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:15.932733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 293/597] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:15.933903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:15.935429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103611-f22e-4528-8b1a-a1def9e6a260 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:15.969462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:15.976569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:15.977379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:15.977722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:15.978110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:15.986043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:15.987899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:16.000992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:16.002147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.003219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:16.004166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 305/598] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:16.005049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.061887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470ea7a-c413-46c7-89cc-444c43a9004e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.110928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.111779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.112221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.112640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.113969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:16.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.116845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.126426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791814e-6630-45b8-a2da-c1e1bab6a862 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.133484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 294/599] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:16.134613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.166947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.172876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.173525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.173804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.174220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.181560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.182977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:16.195451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:16.196499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:16.197495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 306/600] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:16.197961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.257839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.269957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50866cc-6e2f-41c9-9315-69a893512b9b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.308698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.309382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.309870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.310251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.310947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:16.311254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.311873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222ce04-1d8c-40cc-826a-aba1f66ae511 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.312725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.325692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 295/601] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:16.326513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.349506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.350423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.350836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.351255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.359173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.360760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2323d67b-fa59-44f1-a4c7-d6641fd20edf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:16.372885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:16.373938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:16.375010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:16.376638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 307/602] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:16.377812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b0db-2652-416a-84d5-0b262897d500 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.378085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b0db-2652-416a-84d5-0b262897d500 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.378355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b0db-2652-416a-84d5-0b262897d500 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.379760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 308/603] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:16.380233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63b0db-2652-416a-84d5-0b262897d500 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.418540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.439371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e4c9-c0fc-4cc2-8d84-ae3cfe50471a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.478032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.479030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.479520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.479989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.480750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:16.481155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.482794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.492666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63b0db-2652-416a-84d5-0b262897d500 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.497930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 296/604] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:16.499045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.508680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.534444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.535086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.535359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.535629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.551094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=8e403816a5674d45b3cefdc34cffcf7d, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.552480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca6ee8-5870-49db-9eb0-3cf35692c5b1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.565500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 309/605] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:16 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:16.567008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.567517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.567984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.588444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:16.590226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:16.629205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395634e6-ae79-487e-9e9a-574bc2170a99 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.678072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 310/606] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:16.679329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.682775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.684327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.684611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.684882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.685618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:16.685926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.687353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.712270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.723025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 297/607] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:16.726935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:16.853498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:16.855787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e93d90-ab9d-4224-b981-d11268ce2df7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.865125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350ec8a-e3e4-409e-a3bc-cdc4d4311df1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:16.905547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.905908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:16.906577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.906906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.907179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/ce964ecd12bd4e3d9d85bf5dc227d35f/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.907435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:16.907892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:16.908348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/170658d3fbf547a09a4dd2bb77ecb01a/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:16.923632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=170658d3fbf547a09a4dd2bb77ecb01a, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.925265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=ce964ecd12bd4e3d9d85bf5dc227d35f, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:16.925666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.926694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:16.948052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 311/608] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:16 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/170658d3fbf547a09a4dd2bb77ecb01a/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:16.971552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 298/609] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:16 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/ce964ecd12bd4e3d9d85bf5dc227d35f/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:17.192910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:17.309298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccc344-a0c6-4d0a-9e83-b23f0adcf18a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:17.350305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:17.351067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:17.351272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:17.351498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:17.364229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-83f44672-d5ef-4358-bb65-f8c94e7ecd21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:17.377278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:17.378289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:17.379105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 312/610] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:17.386200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:17.457338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ea24-cad9-4dd9-8324-3a03ce7207dc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:17.457522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ea24-cad9-4dd9-8324-3a03ce7207dc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:17.457800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ea24-cad9-4dd9-8324-3a03ce7207dc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:17.459284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 313/611] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:17.470655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:17.471660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:17.471940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:17.472211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:17.490758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:17.511579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:17.513575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:17.551654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274428a-4417-479f-99fc-3f480fe475b8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:17.598568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65856546-b563-456d-9820-74def955c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:17.600321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856546-b563-456d-9820-74def955c0dd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:17.601302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856546-b563-456d-9820-74def955c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:17.602263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856546-b563-456d-9820-74def955c0dd None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:17.603128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856546-b563-456d-9820-74def955c0dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:17.604585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856546-b563-456d-9820-74def955c0dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:17.607239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856546-b563-456d-9820-74def955c0dd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:17.626359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 299/612] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:17.632600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65856546-b563-456d-9820-74def955c0dd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:17.668787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 314/613] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:17.756634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65856546-b563-456d-9820-74def955c0dd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:17.774771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:17.799047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:17.799675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:17.799949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:17.800217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:17.815106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:17.816597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:17.831828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:17.832804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:17.833726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:17.833726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:17.833726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:17.833726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 300/614] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:17.838451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:17.992830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315d4e2-314b-4a16-826f-ca408dbdaa5e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.057287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:18.061969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.063715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.064333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.064611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.064882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.065572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:18.065876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.067246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.079484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 315/615] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:18.090355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.206044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75c90-328e-4605-bfc6-3964d87ace53 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.265327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21636c65-e0a8-4729-b848-5c9faf2dd797 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.278792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.279511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.279800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.280060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.295886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.297355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:18.320227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:18.321173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:18.322102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:18.329108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 316/616] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:18.330230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.334276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.334902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.335203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.335490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:18.352979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:18.353849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:18.354718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 301/617] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:18.360277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.413310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:18.436588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:18.504841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3497a0c-e192-4af0-a7d1-a179c6afe4aa None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.536633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682315b-44e9-4004-a92f-43af00587809 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.539496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.540105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.540378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.540644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.541330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:18.541629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.543001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.552284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 302/618] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:18.561981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.629206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.629706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.630160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.630551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.631300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:18.631604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.633696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.682053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 317/619] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:18.683874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.717361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:18.739533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564a9dc-ec1e-49fd-b3da-d1e903569486 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.781730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.782626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.783014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.783326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.797777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.799221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:18.815840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:18.816820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:18.817733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:18.817733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:18.817733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:18.821505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 303/620] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:18.822822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:18.863833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dcb1f-e3aa-473f-ba70-b8719cc273b7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.884794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:18.937494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:18.938124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:18.938400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:18.938671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:18.949690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:18.952093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:18.968019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd623d5-1993-4e8c-ad27-5fc082703d0f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:18.976381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:18.978343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:18.979488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:18.979488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:18.979488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:18.979488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 318/621] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:18.984095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.028972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.028972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.028972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.028972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.029568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:19.029568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.030012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.039144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.049421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 304/622] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:19.051203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.191050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea830-3c9d-4dc4-9a97-078c9809a3a1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.212527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd72997-92b4-4b37-8be2-22d4586e8387 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.251755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.252389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.252663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.252930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.260644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.262132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:19.275598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:19.276625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 07:11:19.277595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:19.281416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 305/623] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:19.283473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.307205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.307809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.308080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.308415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.308997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:19.309336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.310878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77e58-8eea-4bea-95cd-3eb4bca76591 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.328034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 319/624] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:19.329399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ad45b-8105-4815-b8c2-82fb598aeb33 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.329678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ad45b-8105-4815-b8c2-82fb598aeb33 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.329948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ad45b-8105-4815-b8c2-82fb598aeb33 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.331434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 320/625] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:19.332133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ad45b-8105-4815-b8c2-82fb598aeb33 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.363401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.365949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.416797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28285c8d-bc86-46e6-9176-ea8f9ff8a23f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.459246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.459767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.460043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.460323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.461103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:19.461251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ad45b-8105-4815-b8c2-82fb598aeb33 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.461693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.462875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afcc43-4954-4198-af02-c5fb897263f2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.488413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 306/626] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:19.488413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.488413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.488413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.504601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.505531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.506008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.506435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:19.506636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.507630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:19.514302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.515975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:19.528002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:19.528970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:19.529990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:19.532188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 321/627] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:19.533248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.592612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 307/628] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:19.593832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.641130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.651321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.667115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c0b0a-bd1c-4666-bebb-0eeb103358ee None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.709805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998a4e-b70d-4015-a4cd-21301146c87a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.713266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.714062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.714485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.714899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.752313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.752928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.753235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.753503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.754187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:19.754485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.755902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b74ea-6f84-445d-8b06-c029b139495b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.767884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 308/629] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:19.769168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16985c2-2f11-4715-b986-aac3cbf6824d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.769452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16985c2-2f11-4715-b986-aac3cbf6824d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.769724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16985c2-2f11-4715-b986-aac3cbf6824d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.772985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 309/630] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:19.772985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dfd6-86cf-40b6-a3f6-b4d5df993890 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.772985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dfd6-86cf-40b6-a3f6-b4d5df993890 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.772985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dfd6-86cf-40b6-a3f6-b4d5df993890 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.774499 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 310/631] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:19.779389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893dfd6-86cf-40b6-a3f6-b4d5df993890 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:19.800145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.801726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dadff-f05b-46c7-b76a-d6a4dc30b0b6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.885688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893dfd6-86cf-40b6-a3f6-b4d5df993890 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:19.886727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.893986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:19.908368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 322/632] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:19.909631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.910006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.910279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.923336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:19.923984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.924259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.924530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/c4389bd42da64dccbf958503dccf87e1/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.929099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:19.930279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:19.938685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=c4389bd42da64dccbf958503dccf87e1, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:19.940100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b03a2-1b66-46a2-bdc3-c40df06ce8a7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:19.953001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 311/633] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:19 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/c4389bd42da64dccbf958503dccf87e1/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:19.954201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:19.954485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:19.954758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:19.976890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:19.978068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:20.024974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 323/634] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:20.024974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.073843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 312/635] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:20.074902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.177437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917ade-dc25-4254-a297-1ff5bd8e48fd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.184986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.201101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.204717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edbd71-c43b-4938-a71d-4765c4755718 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.229322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.230398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.230675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.230943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.244184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:20.244665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.245343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.245615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.245876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:20.249792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.250715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:20.251706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 324/636] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.258567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.266586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.268442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:20.287881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.288898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:20.290169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:20.290169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:20.290169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:20.296845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 313/637] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.298567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.407392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877aa85-1af2-402d-8fa2-0d56cc7f81af None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.414814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4350-c9bc-4201-a31e-5d6ed7727219 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.445906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.446693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.446813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.447083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.447663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:20.447966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.449353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.451333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.451931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.452202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.452465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.453155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:20.453461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.454816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.457960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.468709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 314/638] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:20.468709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.469076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 325/639] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:20.470378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.511560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.579808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b553-304b-48e2-b280-546b81a8849e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.584018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501ac3e-7c1f-4e81-86e0-8776a576bf1a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.618387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.619011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.619286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.619557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.619892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.620509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.620784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.621081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.630736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:20.631822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:20.635505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.636523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.638628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:20.639262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.640120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:20.641008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:20.642731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 326/640] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.642731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.643059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 315/641] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.643555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.714918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.745641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.755048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd92ffe-8931-45c7-900d-4fce29b16463 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.762873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e824ea-5aff-4b5a-b1d2-fccc8094c9cc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.788901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.789542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.789818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.790082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.800096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.800706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.801002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.801274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.801390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.802791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.808350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.811486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:20.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.821564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:20.822856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:20.822856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:20.822856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:20.823286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:20.824109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 316/642] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.831087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.831632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 327/643] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:20.831632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.936586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc2cc-5e4c-497b-9cb3-917a86fca217 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.938721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16230b-82e3-4b92-b5d3-8acd55e06696 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:20.956466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:20.971430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.972036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.972313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.972582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.973338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:20.973587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.975079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.975550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:20.976231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:20.976575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:20.976918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:20.977685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:20.978061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:20.979520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:20.989959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 328/644] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:20.989959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:20.995406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 317/645] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.001300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.002573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.097527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a582a1-c9ef-4a85-a7b8-63c1e5b40eb3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.108233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6d33-9655-46f8-9938-b5e3103a222e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.131974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.132647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.132921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.133220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.133873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.134173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.135529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.144579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.145467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.145733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.146003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.146684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.146985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.148392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.151563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 329/646] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.152261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.166162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 318/647] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.166162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.212451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.226207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.260981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee922d1-cb0b-4477-842a-4fb7dd5d87b0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.275194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390b2c3-041c-4bc0-b5f3-937cf0496a8d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.297825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.298444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.298714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.298983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.310784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.313258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.314441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.315059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.315457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.315598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.327143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.328569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:21.331542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:21.332666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:21.333779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:21.333779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:21.333779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:21.339453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 330/648] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:21.339453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:21.348503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:21.349629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:21.350719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:21.350719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:21.350719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:21.355782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 319/649] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:21.355782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.447829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960394-3c76-43d6-89ce-3930e5f5a504 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.454328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.464141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190db29d-7d7c-4206-82da-8a1c7993eb92 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.468750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.484044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.484655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.484914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.485222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.492901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.494369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.500620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.501247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.501523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.501791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.502505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=20ca1a72134341ed9dbe8b58c339873e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.502836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.504242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:21.506178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:21.507083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 331/650] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:21.508000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.525368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 320/651] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/role_assignments?user.id=20ca1a72134341ed9dbe8b58c339873e&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.525368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.642685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af1ba2-ca46-4827-b7ce-12302093be7e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.662506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df76f51-5e82-413b-a667-e6486daa125b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.685299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.686523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.686904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.687658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.688434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.688853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.694059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.717006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 332/652] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.717006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.721043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.722470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.722750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.723070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.723733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.724034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.725460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.743203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 321/653] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.744799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.768851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.779072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:21.862730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcf89b-e600-462d-8729-ae51b88de024 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.878068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad138b00-237a-468f-9f0b-e375112524ad None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:21.908169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.909121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.909520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.910300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.911411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:21.911821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.914118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.918694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:21.919585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:21.920039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:21.920483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:21.928607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:21.930416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:21.935402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 333/654] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:21.936439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:21.944510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:21.945507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:11:21.946424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:21.954301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 322/655] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:21.955479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.041996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.062295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.069936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0749-db33-465f-b9bb-79535e08162a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.071642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976883f-c663-45b9-941a-7d2c78315637 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.113575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.114325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.114456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.114808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.115290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.115584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.116134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.116603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.123807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.125424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:22.138546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:22.139732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 07:11:22.140882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:22.142580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 334/656] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:22.143965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.210905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.212923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dd697-fb28-4be5-91e5-4534fe7677c0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.256246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84db97-d193-4c05-b324-3bfaf7d93740 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.292980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.294097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.294097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.294563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.295396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:22.295806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.297386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.302046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.322744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 335/657] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:22.322744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.338679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 323/658] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:22.340780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7ec3-165b-4697-9331-1cb1d3705167 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.341528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7ec3-165b-4697-9331-1cb1d3705167 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.341935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7ec3-165b-4697-9331-1cb1d3705167 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.343413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 324/659] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:22.344260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.344408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7ec3-165b-4697-9331-1cb1d3705167 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.439403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950814f-9072-4b80-92d0-b1d5c5958c7f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.458459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7ec3-165b-4697-9331-1cb1d3705167 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.475486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.476018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.476299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.476580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.477342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:22.477655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.479080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.495779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.496380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.497289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.497574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.497854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 336/660] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:22.498754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.553271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.579086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.583196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.585463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.606475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60fb13-c686-4c8e-b033-8b4ddf96da0b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.642330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.642953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.643232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.643505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.654384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.655830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c675197c-3203-43a3-8195-a89ff9c43340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:22.670946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:22.671894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:22.673444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:22.673444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:22.673444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:22.673444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 337/661] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:22.673677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2b371-e109-48e4-a569-85306df24701 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.673896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2b371-e109-48e4-a569-85306df24701 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.674049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2b371-e109-48e4-a569-85306df24701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.694627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-44f2b371-e109-48e4-a569-85306df24701 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:22.695829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44f2b371-e109-48e4-a569-85306df24701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:22.704999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 325/662] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:22.706009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.770416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 338/663] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:22.771303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2b371-e109-48e4-a569-85306df24701 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:22.791210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.823637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611e536-fd14-431b-b6ce-57f5560ecf2e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.835427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:22.862800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.863645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.863913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.864180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.895506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2b371-e109-48e4-a569-85306df24701 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:22.942466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.943288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:22.943909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.944353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:22.944829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:22.945317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:22.946400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:22.947719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:22.949364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:22.964645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 339/664] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:22.965793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.045317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.066015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 326/665] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:23.067238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.090196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa20c4-88b8-4eae-b3b3-7305ad6cf38d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.110032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.130136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.130762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.131042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.131317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.143269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.144741 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:23.160544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:23.161700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:23.162579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:23.162579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:23.162579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:23.162579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 340/666] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:23.163302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.179251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d4f26-aaec-49e6-8d90-27c8f447d7f5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.219583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.220204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.220482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.220743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.227424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.228873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:23.240719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:23.241788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:23.242666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 327/667] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:23.243165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.273809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252cd7-0e21-4c3a-906b-5fc9eb610fbb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.281244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.310667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.311522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.311959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.312233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.323230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:23.328483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:23.329470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:23.330294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 341/668] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:23.331821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.356644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ef572-fb8d-4dd2-90ff-8940c5f79128 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.360463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.394725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.395537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.395650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.395860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.406783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.408224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:23.423328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:23.424439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:23.425547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:23.425547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:23.425547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:23.429090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 328/669] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:23.429829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.438769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a41ab-6d40-4b92-8b81-2c09f4b28a81 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.477253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.477849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.478128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.478461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.479103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:23.479450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.480823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.493954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 342/670] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:23.494611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.527300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.542057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3530-b665-46e4-bcba-d2c7c58a34ef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.582815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.583654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.584101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.584281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.584559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.585373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:23.585752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.588041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.597124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534698-e640-4e06-9e0a-db267a094551 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.607784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 329/671] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:23.607784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.638275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.639004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.639146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.639411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.640073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:23.640371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.641802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.650011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 343/672] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:23.655557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.719776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccbab-5776-498c-84b6-fdf214552284 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.760324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.760978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.761238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.761503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.762159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:23.763251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.764641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.770004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce150a-d769-4846-bbc7-5acd26acf857 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.776882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 330/673] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:23.777890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.790284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.814192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.815336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.816566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.817012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.825893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.827693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:23.842956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:23.844016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:23.845142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:23.847662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 344/674] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:23.848587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.865330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:23.883578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89885bc-d41f-4147-a97a-463403ab4017 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.917069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:23.917663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:23.917939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:23.918210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:23.919749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:23.920059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:23.921501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:23.927046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 331/675] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:23.927825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:23.959021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953179-3371-4ab2-83c4-9e09cfa72c5d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:23.999995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.000634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.000911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.001217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.007650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.013638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.015107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:24.030984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:24.032025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:24.032928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:24.032928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:24.032928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:24.032928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 345/676] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:24.033862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.048530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97d6c-5079-4c69-be85-1732b055e652 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.087765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.088416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.088694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.088990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.096783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.098194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.105466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-67369bb6-a681-443a-8758-94a80e23b5ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:24.113779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:24.114688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 332/677] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5b6-6939-4f71-9eb0-964b2e57c0e0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.118795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5b6-6939-4f71-9eb0-964b2e57c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.119596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5b6-6939-4f71-9eb0-964b2e57c0e0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.119596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 333/678] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:24.119596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966b5b6-6939-4f71-9eb0-964b2e57c0e0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.158526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63dc75-4ca2-42db-8148-62035e0dfb05 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.200596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.201292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.201572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.201842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.202734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.202841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.204270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.216677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 346/679] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.217527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.250110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966b5b6-6939-4f71-9eb0-964b2e57c0e0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.285275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.292113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.292732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.293035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.293309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.294823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.295227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.296697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3c4a-ed2b-4c94-88cf-490d08abeb80 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.311100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 334/680] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.312437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.312714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.313005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.328962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3dda-5df7-4464-891b-54358f9aefdd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.331952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:24.333035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:24.354524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.368843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.369689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.370135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.370544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.371379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.372340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.373918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.385400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 347/681] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.386193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.410884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 335/682] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:24.411977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.509884 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698970e-00cc-49d4-b37d-261a5cfeedbc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.547771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c7d8-29c3-45f5-bead-c0749633a10c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.551832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.552625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.553100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.553538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.554529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=ce964ecd12bd4e3d9d85bf5dc227d35f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.555006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.556851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.564061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 348/683] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/role_assignments?user.id=ce964ecd12bd4e3d9d85bf5dc227d35f&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.565103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.588364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.617250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.619229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.623358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.623898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.639925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.641721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.649614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:24.667038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:24.668081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:24.669579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:24.672177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 336/684] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:24.679424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.759545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a6a57-d4ef-41a6-b992-a849ed643e79 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.807776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.808716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.810621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.811167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.812202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=170658d3fbf547a09a4dd2bb77ecb01a) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.812655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.814929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.822184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097874-9157-473f-8c18-92beae1a3993 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:24.823103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 349/685] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/role_assignments?user.id=170658d3fbf547a09a4dd2bb77ecb01a&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.824040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.882895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:24.883608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:24.883980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:24.884347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:24.885260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:24.885666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:24.887797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:24.912819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 337/686] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:24.918645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:24.975805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:24.986864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91295ac3-a93e-4d2d-af7a-a8904b419c48 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.002870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:25.056619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.057285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.057559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.060463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.075449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571790eb-5933-4c1a-b844-db0260e88f71 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.076426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.079349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:25.098464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:25.099444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:25.100310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:25.100310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:25.100310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:25.102325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 350/687] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:25.109778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.122238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.123240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.123240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.123426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.201936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.203307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.247665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820456-478c-41f4-bc45-7dd6080de45b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.251099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:25.287353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.288363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.288834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.289315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.290153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:25.290717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.292337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.305965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 351/688] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:25.312309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.317503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 338/689] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:11:25.323610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.344575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:25.416172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33acfe-df42-4712-b946-f4738739eab5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.426743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810747-cfa4-406b-a636-70b365e750e2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.452100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.453050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.453545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.454036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.454863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.456500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.462451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.463069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.463341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.463607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.472698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:25.479478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.479478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:25.498166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:25.499124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:25.500031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:25.500031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:25.500031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:25.500031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 339/690] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:25.502993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 352/691] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:11:25 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:25.505100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.622399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068a1d-6e01-4422-b7d5-5892ad898c2e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.660275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.660877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.661177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.661444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.662084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:25.662381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.663733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.678286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 340/692] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:25.686075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.822021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:25.847920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111cf0c-346c-416d-9768-80fd0f2c204f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:25.885987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:25.886616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:25.886887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:25.887189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:25.894849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:25.896285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:25.907495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:25.908381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:25.909447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 353/693] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:25.914293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:25.974958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:26.025278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405586-f6ad-4510-a5ac-dce94a25569b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:26.063745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:26.064354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:26.064626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:26.064961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:26.065660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:26.065958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:26.067388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ffadf-d963-4d84-b04d-703f6a37350e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:26.080701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 341/694] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:26.086676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:26.204453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdd8e-b702-4395-8a0b-4d76328fe901 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:26.241484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:26.242165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:26.242466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:26.242733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:26.243541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=c4389bd42da64dccbf958503dccf87e1) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:26.243844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:26.245374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0746c-274a-443e-ab31-05c529fd2e78 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:26.252645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 354/695] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:26 2022] GET /identity/v3/role_assignments?user.id=c4389bd42da64dccbf958503dccf87e1&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:26.340506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e5e-10c7-43d8-aefd-7b2a99b9dc5e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:26.340837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e5e-10c7-43d8-aefd-7b2a99b9dc5e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:26.341072 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e5e-10c7-43d8-aefd-7b2a99b9dc5e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:26.342876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 342/696] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:26.348356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:26.348640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:26.348902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:26.358787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:26.376234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:26.377456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:26.476118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 355/697] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:26.768644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939e5e-10c7-43d8-aefd-7b2a99b9dc5e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:26.908037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939e5e-10c7-43d8-aefd-7b2a99b9dc5e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:26.928835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:26.959348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:26.960196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:26.960611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:26.961042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:26.977814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:26.983656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:26.984562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:26.985464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 343/698] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:26.991905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.141662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428dcf3-fd09-4a74-8901-567fbbb69d8b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:27.181848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:27.186948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:27.187602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.187882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.188162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.191139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:27.191460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:27.192842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07c23-a9c5-40e1-813b-7fb574886170 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:27.202576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 356/699] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:27.208030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.293275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb473f-4671-41e3-b194-7bb81d50535a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.293584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb473f-4671-41e3-b194-7bb81d50535a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.293925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb473f-4671-41e3-b194-7bb81d50535a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.296459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 357/700] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:27.310631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.310917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.311186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.342108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:27.343297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:27.350128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd143e-7a51-4e8d-ad97-6c165c1b09ad None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:27.384740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:27.385397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.385678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.385952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.397330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:27.398793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:27.413568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:27.414572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:27.415460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:27.415460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:27.415460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:27.419615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 344/701] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:27.420900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.467448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:27.511075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 358/702] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:27.542527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f816dd-3185-4755-9694-d7ce885534fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:27.579504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:27.580028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.580307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.580580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.581253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:27.581557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:27.582911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:27.596412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 345/703] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:27.601727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.764548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7ef79-e0ae-4dc1-a792-3517414c3ed1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:27.793130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:27.808332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:27.809167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:27.809653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:27.810134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:27.821683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:27.823153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:27.840853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:27.841829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:27.842855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 359/704] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:27.848527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.849175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:27.993235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.015953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924608-a061-4338-a399-01107e3d8ee1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.020161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b4746-43f3-4713-88bd-0b94aa77b0f5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.053145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.053756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.054246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.054522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.055338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:28.055906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.057378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ab5f7-dd58-47af-b942-0a3b491e9e4c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.064044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.064676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.064971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.065263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.066148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.067771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.079368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 346/705] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:28.080728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc564-f065-4acc-941a-d2a76da0b0c9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.081105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc564-f065-4acc-941a-d2a76da0b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.081472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc564-f065-4acc-941a-d2a76da0b0c9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.086928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 347/706] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:28.087788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:28.088764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:28.090143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.091859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.091859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.091859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.096017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 360/707] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 07:11:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:28.096017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.113753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:28.115057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:28.152891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.194782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 348/708] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:28.195749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.230759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349da7e-cc63-4744-96a4-b9d9c84a0934 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.285793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.305111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.305976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.306252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.306522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.341254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aeeab-cc02-49e6-822e-8c49f5e160c6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.398495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.399027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.399305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.399579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.400454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:28.400767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.402312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.407645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 349/709] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:28.418874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.432362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.434634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.539598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.594584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593664e8-2774-4ef6-8ab9-23f0e70c75c0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.595187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 361/710] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:28.595908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.637609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.638654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.638942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.639206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.639962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:28.640265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.641644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bf35b-33aa-4073-92d5-f1f61e589286 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.644992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.648156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 350/711] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:28.649793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b34555-2b00-49bd-9778-e7022cee1c50 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.650130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b34555-2b00-49bd-9778-e7022cee1c50 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.650504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b34555-2b00-49bd-9778-e7022cee1c50 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.657235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 351/712] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:28.658355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b34555-2b00-49bd-9778-e7022cee1c50 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.714981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc817-d759-481d-bd6c-5a1b9cea5e55 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.756611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.757313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.757588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.757858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] PATH_INFO: `/v3/services/network` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.758451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.759963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4ce4996e-015d-4791-bffb-6391142fa0dd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:28.770873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:28.771981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:28.772807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:28.777280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 362/713] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:28.778887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.779180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.779460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.792729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b34555-2b00-49bd-9778-e7022cee1c50 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:28.800036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:28.805007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:28.815212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:28.838275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:28.839250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:28.839712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:28.840265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:28.852518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:28.854244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:28.869640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:28.870897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:28.872390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:28.872390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:28.872390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:28.874173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 352/714] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:28.876571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.902320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 363/715] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:28.903148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:28.948996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.051093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99562487-e857-48dc-9c8a-da3b6d08edb0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.054382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f76af-c197-4339-98bb-c7d17309d5b1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.097710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.098453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.098729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.098997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.099703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.100010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.101467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.117157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 364/716] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.123151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.127800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.140960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.141599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.141874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.142190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.142821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] RBAC: Adding query filter params (name=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.143117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.144482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.156720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 353/717] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.163875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.281589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67192-b3ba-477f-90aa-c7e37b4d0683 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.305413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26fd09-b0e6-473e-9d70-d6cf2f25341a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.316833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.317487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.317759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.318025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.325541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.326311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.326987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-761586f7-8334-40f0-99d7-1df8b9153f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:29.339199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:29.340864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:29.341993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:29.345461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 365/718] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:29.346713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e3c46-20ae-4e89-b816-429967d5a4fb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.346993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e3c46-20ae-4e89-b816-429967d5a4fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.347260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e3c46-20ae-4e89-b816-429967d5a4fb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.352695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 366/719] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:29.352888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.352888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.353405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e3c46-20ae-4e89-b816-429967d5a4fb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.353556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.353556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.353556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] RBAC: Adding query filter params (type=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.353556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.353556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.356920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 354/720] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.362042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.424431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.476751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e3c46-20ae-4e89-b816-429967d5a4fb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.513853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba312e4c-9bf2-4490-9b45-12c1a220a659 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.531553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.532191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.532464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.532731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.547272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-af510a59-f83b-4096-acff-30dd87125f56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:29.552137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:29.553379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:29.554265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:29.555050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 367/721] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:29.555969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.556250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.556522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.566896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.567741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.568187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.568601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.569484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.569794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.571216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.578634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:29.579818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:29.586713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 355/722] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.587571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.640718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.669126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 368/723] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:29.670161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.745301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fd0f-1408-4f82-904b-048bdb915967 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.763951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:29.821659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.822334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.822608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.822876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.823600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.823902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.825832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.827592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be26f1-f316-4e5c-9aa5-246139d05d18 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.842036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 356/724] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.847332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:29.899755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:29.900635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:29.901089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:29.901500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:29.902359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:29.902840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:29.904419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:29.919745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 369/725] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:29.979271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa9fe-1c63-4b1c-ae25-feda6d4054f8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:29.980688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.011620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.026478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.027120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.027394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.027703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.040645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.042149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:30.060052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:30.061079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:30.061981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:30.061981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:30.061981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:30.061981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 357/726] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:30.068024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.201943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f687a6c-3f40-4f2a-839e-14f60cd56656 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.222964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.245443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.245992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.246269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.247267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.259273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:30.266652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:30.267581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:30.268394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 370/727] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:30.270767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848f1-6a75-4d96-83e6-ff5fa0d17aeb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.276239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.355742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.356969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.357257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.357526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.358168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:30.358467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.359847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.397759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 358/728] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:30.406027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.414803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7f65-458f-412a-81c0-278b164dbbb6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.448530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.456071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.456725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.457029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.457305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.458070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:30.458376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.459738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.469419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 371/729] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:30.475284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.547657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.548584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca1c34-470e-47bc-aaac-74c07f793976 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.583767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.584391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.584667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.584935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.592865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.594335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:30.606663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:30.607707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:30.608650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:30.612452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 359/730] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:30.613694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.676365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc622ef1-9e08-4f80-a673-562b2a5105d6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.705349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.729613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44427e-ce5d-4b42-aac8-940eb60ba86a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.764999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.765918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.766339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.766781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.767659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:30.768107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.769693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.792834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.792834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.792834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.792834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:30.793439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 360/731] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:30.794822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.842074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:30.843461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:30.874838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:30.876073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:30.877447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:30.877447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:30.877447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:30.877447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 372/732] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:30.885877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:30.902390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f1d31-9fde-4749-99e6-1c14524d7a07 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:30.936639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:30.940877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:30.941700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:30.942123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:30.942533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.044271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.047866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83090220-13c9-42f8-bcf7-b332b5a39081 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.135389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.150586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead9c3-2aba-424b-8d64-9a2922eb06a3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.180411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 361/733] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 07:11:31.182889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4820a-10ed-4b22-a95d-6489968992d5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.183324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4820a-10ed-4b22-a95d-6489968992d5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.183739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4820a-10ed-4b22-a95d-6489968992d5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.186506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 362/734] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:31.187525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4820a-10ed-4b22-a95d-6489968992d5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.198941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.200388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.200671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.201080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.201780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:31.202091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.203809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735da13f-bb8b-4c8e-872d-f97b7d76fd6e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.218463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 373/735] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:31.219710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.219987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.220253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.232809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.243753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:31.245363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:31.312859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4820a-10ed-4b22-a95d-6489968992d5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.333546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 374/736] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 07:11:31.334575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.359704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.361200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.361488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.363018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.386573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.390771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:31.417171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:31.418146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:31.420560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:31.420560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:31.420560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:31.420560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 363/737] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:31.426555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.453677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.476606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedcc65-8e5d-445e-834f-3f9e0c4698eb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.514059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.514778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.515167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.515542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.523653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.525581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:31.539176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:31.540159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:31.541417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 375/738] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:31.547599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.655129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3a91c-d19a-4042-bc31-292f95757fdc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.669609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.694444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c43d-3a86-4bcf-8909-b80bcac9694d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.698852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.698852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.698852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.698852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.699256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:31.700225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.701814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.713336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.723383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 376/739] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:31.724349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.739164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.740062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.740566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.741070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.741905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:31.742405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.743978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.761022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 364/740] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:31.762878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.832669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52842-5528-4b2a-ac06-929e9b4250f4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.865890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a022f1-e8f2-4bde-9265-95c904c1853a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:31.867958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.868732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.869029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.869298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.869942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:31.871290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:31.897004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 377/741] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:11:31 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:31.897004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:31.919841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:31.920431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:31.920710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:31.921016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:31.935943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:31.981075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.035549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67b6f8-f2ea-4609-99db-0ddba3fdcd05 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.042097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.045107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.079994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.080977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.081535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.081816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.090034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.091717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ea12a8ad-ecb3-49f5-a77f-fe9c88c444da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:32.104188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:32.105179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:32.106742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 378/742] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:32.108000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccbb3-469f-4980-a3d9-3d111301cc5e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.108000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccbb3-469f-4980-a3d9-3d111301cc5e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.108176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccbb3-469f-4980-a3d9-3d111301cc5e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.112748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 379/743] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:32.116705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.117586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.118325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.139469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:32.140665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:32.172209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 365/744] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 07:11:32.173102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.217097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 380/745] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 07:11:32.217974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.227189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.281765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.297907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4782bec-9a2e-42df-a570-c7ee0b90cd11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.338005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.338663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.338967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.339594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.340058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:32.340969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.342456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.355187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9cf4-320d-4f35-8b59-9d522eb5856e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.360356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 366/746] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:32.362040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.409660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.410655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.411158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.411657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.433647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.433647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:32.448378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:32.450157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:32.451120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:32.451120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:32.451120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:32.453183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 381/747] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:32.458269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.521833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.525586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e436b-3cc6-4566-9459-638fad4938d3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.567930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.569397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.569397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.569397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.570334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=a3ccebac846245ee915636757eb921f7) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:32.570737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.572301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a31c6-0434-4ae7-970f-140973542d41 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.585056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 367/748] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:32 2022] GET /identity/v3/role_assignments?user.id=a3ccebac846245ee915636757eb921f7&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:32.587619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee95f-71f5-4c9a-909e-df15d8887cc6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.588973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee95f-71f5-4c9a-909e-df15d8887cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.589572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee95f-71f5-4c9a-909e-df15d8887cc6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.593392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 368/749] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:32.599317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee95f-71f5-4c9a-909e-df15d8887cc6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.616767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.623442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ebe3-f50d-446a-9664-3e93831ddcd3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.677043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.677639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.677916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.678185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.678844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:32.679144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.680798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e910-6872-4e00-84e6-7dcf335a676f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.703534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 382/750] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:32.703534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.703534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.703534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.728039 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:32.728666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee95f-71f5-4c9a-909e-df15d8887cc6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:32.729565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:32.768229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:32.769107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:32.769569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:32.769977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:32.770766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:32.772294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:32.790931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 369/751] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:11:32 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:32.791759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:32.850427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 383/752] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:32.851035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:32.970211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752ea0-4b6f-4f2f-b83a-aca273b9bf64 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:33.019945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:33.020583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:33.020861 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:33.021162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:33.028863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:33.030322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:33.041617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:33.042493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:33.043885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 370/753] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:33.048564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:33.156396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:33.174367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e5c8-84a0-4774-8fec-ab1cfcf092ce None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:33.203872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:33.222627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:33.223244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:33.223439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:33.223709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:33.225391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:33.225702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:33.227153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:33.249377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 384/754] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:33.254314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:33.340598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:33.367115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df965b32-df9f-46ad-b915-f370b7635afc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:33.429994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:33.430876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:33.431153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:33.431424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:33.433706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb50a5-0d2a-4cac-bd8b-5af1cd31cda8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:33.453843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:33.459301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:33.460262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:33.461516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 371/755] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:33.469715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:33.508222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:33.508749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:33.509056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:33.509391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:33.510119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=c4389bd42da64dccbf958503dccf87e1) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:33.510440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:33.511860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:33.519465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 385/756] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:33 2022] GET /identity/v3/role_assignments?user.id=c4389bd42da64dccbf958503dccf87e1&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:33.677422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17552cae-90b8-4016-a8c2-4ecc273cdc17 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:33.722355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:33.722956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:33.723233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:33.723501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:33.724154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:33.724451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:33.725857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:33.734768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 372/757] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:33.739997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:33.878509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:34.006170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ab7be-eceb-4965-8aec-d98ab75f9169 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:34.057216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:34.057861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.058133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.058401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.070387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:34.071836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:34.087001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:34.087976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:34.088878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:34.088878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:34.088878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:34.088878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 386/758] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:34.094529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:34.104885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:34.221562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4887913-47a7-4295-bdc5-6d8e24838803 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:34.267013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:34.267623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.267895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.268160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.268800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:34.269128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:34.270553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:34.290312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 373/759] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:34.297223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:34.391649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:34.435921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89bb54-cd8b-4409-9f6a-1a7c9c8cbfde None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:34.481996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:34.482577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.482858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.483130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.491297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:34.492782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6e296ac6-1963-43e0-958d-84d1dfd0779e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:34.509470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:34.510442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:34.511366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 387/760] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:34.517171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:34.584301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e62f41-9bcd-4334-b91c-7f9bdbd0cf56 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.584621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e62f41-9bcd-4334-b91c-7f9bdbd0cf56 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.584851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e62f41-9bcd-4334-b91c-7f9bdbd0cf56 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.586386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 388/761] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:34.592715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.593047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.593284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.629780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:34.631497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:34.633702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:34.689110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e1e46-79ac-4d6f-a263-a194e064d11e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:34.733056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:34.734020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.735325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.735812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.738430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:34.738945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:34.740567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:34.776209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 374/762] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:34.780838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 389/763] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:34.785921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:34.892675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dbbff-9047-453a-8258-94fcb6511ab0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:34.920348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:34.938522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:34.939148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:34.939416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:34.939677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:34.949971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:34.951407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:34.971744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:34.972675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:34.973593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:34.973593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:34.973593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:34.977374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 375/764] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:34.978610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.065197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:35.129457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a85a7-9942-4687-ae22-6e97d5df3f7b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.135749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.185549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.186174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.186451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.186721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.187390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:35.187774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.189324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.211700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 390/765] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:35.217617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.325908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fcf72-204c-4618-a9bd-7e34f31389bc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.363434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.363977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbe96c-8874-4a71-83b4-91fb4056bb9c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.364418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.364418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.364670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.365252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.367768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.379162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a1faed6c-3676-434d-85fc-c9ec43189a76 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:35.382779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:35.383758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:35.384605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:35.385442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 376/766] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:35.386406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7bcb-c10e-48f4-baa6-071328c5c79e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.386821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7bcb-c10e-48f4-baa6-071328c5c79e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.386821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d7bcb-c10e-48f4-baa6-071328c5c79e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.389453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 377/767] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:35.390601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d7bcb-c10e-48f4-baa6-071328c5c79e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.410784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.411426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.411800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.412074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.420223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.421904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-80540980-ba5c-4517-8091-82ec701538b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:35.435270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:35.437333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:35.439443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:35.443603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 391/768] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:35.443810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:35.444009 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.445000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.445485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.465796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:35.467255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:35.498707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d7bcb-c10e-48f4-baa6-071328c5c79e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.533362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.534202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.534620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.535055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.535815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:35.536255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.537791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.547808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 378/769] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:35.548891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.549264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 392/770] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:35.550944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.654468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b85e2-75c7-45d4-8087-3a120301ec59 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.662192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:35.701310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.702200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.702617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.703041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.703875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:35.704328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.707724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b536-4fb8-48e6-96a5-39725092fe7f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.732301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 379/771] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:35.734992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4ca4-1e0e-48c0-ac28-c70aa20c4ff8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.736730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4ca4-1e0e-48c0-ac28-c70aa20c4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.738391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4ca4-1e0e-48c0-ac28-c70aa20c4ff8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.739741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 380/772] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:35.741929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d4ca4-1e0e-48c0-ac28-c70aa20c4ff8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.750094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e68178-37c0-4d83-86d2-a5f70c8a7cd4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.751959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:35.791853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.792842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.794084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.794664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.795640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:35.796180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.797936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36325d8-99b8-4e48-a29f-7740bad29b3c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.804990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 393/773] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:35.806800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.807291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.807784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.829969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:35.832017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:35.859696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d4ca4-1e0e-48c0-ac28-c70aa20c4ff8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:35.895342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:35.896259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:35.896259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:35.896518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:35.898433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=8e403816a5674d45b3cefdc34cffcf7d, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:35.899845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:35.921057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 381/774] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:35 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:35.921057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.941076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 394/775] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:35.941971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:35.952335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.037287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ad784-f0ea-4c17-994f-d61925b72cde None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.076428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.077074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.077352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.077615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.078320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] RBAC: Adding query filter params (interface=public, service_id=bc18c3055b854bbc8077c3f9fd7bbce1, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:36.078624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.079750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.079984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.087801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c72d35-f536-417a-bd2f-f920a94bac1e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.093054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 382/776] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/endpoints?service_id=bc18c3055b854bbc8077c3f9fd7bbce1&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:36.095187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.145904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.146576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.146847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.147112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.172433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:36.178868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:36.181123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:36.182855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:36.185629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 395/777] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:36.191591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.237586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.252546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c83273-e1bc-44aa-a3a1-52be4c29c754 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.295642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b42bb-455c-4820-a562-72695955784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.296256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b42bb-455c-4820-a562-72695955784c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.296532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b42bb-455c-4820-a562-72695955784c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.296801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b42bb-455c-4820-a562-72695955784c None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.297451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b42bb-455c-4820-a562-72695955784c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.298828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b42bb-455c-4820-a562-72695955784c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.311600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 383/778] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:36.312818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c851c-1dcc-4aed-bc7a-bde396b8f42f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.313137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c851c-1dcc-4aed-bc7a-bde396b8f42f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.313411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c851c-1dcc-4aed-bc7a-bde396b8f42f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.315071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 384/779] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:36.321978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.322518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.323155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.349260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c47da-1635-4ade-be33-b263f547e3dc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.353217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:36.355082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:36.396529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.397382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.397731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.398074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.398831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:36.399232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.400744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.413314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 396/780] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:36.414262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.440056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 385/781] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:36.441339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.442372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.547247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.569917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2900e7-4375-4719-9adc-d056fd5de505 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.600343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46142f-6616-4821-bb67-1e9af0684fd6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.618615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.619507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.619944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.620353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.641006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.641640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.641914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.642180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] PATH_INFO: `/v3/services/network` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.642763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.644119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.647735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.649284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:36.662947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:36.663962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:36.665389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 397/782] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:36.673323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:36.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:36.677196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:36.679311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:36.679311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:36.679311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:36.691304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 386/783] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:36.692151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.795758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.821552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011e5d0-e4c9-4657-9a53-c098b2e30ecc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.840928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdafb4-bfcd-4e55-9448-946a2cd0c7f4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:36.868054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.868959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.869397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.869685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.870485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] RBAC: Adding query filter params (name=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:36.870791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.872831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.879738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:36.880429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:36.880770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:36.881149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:36.881859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:36.882227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:36.883406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 398/784] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:36.883702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:36.884309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:36.885399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:36.899843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 387/785] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:36.900971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.011710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eaa861-84c2-4c5e-b6c9-6d09c7be900d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.016907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41e8fc-61b6-446a-a6b3-5cbec8d96f73 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.057110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.057369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.057916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.058194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.058467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.058908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.059346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:37.059537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.059641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.060147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.061063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.069108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 388/786] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:37.070224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.080493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:37.086041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:37.086947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:37.087764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:37.090856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 399/787] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:37.091958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.101574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.148262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.202504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635f598-d22e-42bf-a758-d44b7a623bd4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.202975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2d30-0e9e-47f2-9b2f-0a5d64b07ed4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.239272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.240270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.240779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.241285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.242212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] RBAC: Adding query filter params (interface=public, service_id=e0c7618da78344a088bb718dbab0490d, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:37.242732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.242903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.243534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.243813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.244086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.245063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.253330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.254536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 400/788] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/endpoints?service_id=e0c7618da78344a088bb718dbab0490d&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:37.254781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.255757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:37.274968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:37.275963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:37.277046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 389/789] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:37.277877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.337895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.375574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf820-5bfc-4041-8bbf-51c83366030f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.406651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649c071-ecd2-4c93-92ef-d7f9091690f7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.425595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.426583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.427648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.428079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.428494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.429357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:37.430296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.432454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.445918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 401/790] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:37.446914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.454153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.454763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.455038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.455305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.455909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.457310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.469370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 390/791] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:37.470299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.573771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b67bf-6a1b-4824-bc5d-2de8b93777d5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.610401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.617364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.618360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.619014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.619607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.620494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:37.622002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.624105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.646209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 402/792] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:37.652057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.742441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70867b-3b75-4538-a56c-616da6668edd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.774353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185acb41-f381-497d-bdad-ac25482ef0cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:37.787389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.788026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.788274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.788538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.800692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.802149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:37.820570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:37.821769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:37.822653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:37.822653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:37.822653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:37.822653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 391/793] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:37.823561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:37.825085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:37.825576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:37.826051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:37.828141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.844772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:37.846264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:37.867277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:37.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:37.869218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:37.869218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:37.869218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:37.869218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 403/794] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:37.875004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:37.879103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.959260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:37.986801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6e46-7730-4efe-9497-a2b4badf07ef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.052068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.052709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.053002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.053274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.053927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:38.054226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.055590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.077537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 392/795] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:38.083167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.084794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1cc65-b19f-40ad-8be1-596860954a33 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.160808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.161839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.162346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.162816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.163661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:38.164168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.165780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.194835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 404/796] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:38.201263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.251031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:38.260791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce60a8-16d9-4fa5-aa45-77c3215cd7d9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.329498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.330057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.330331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.330600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.345724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.347194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:38.366385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:38.367394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:38.368316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 393/797] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:38.374762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.390462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d5e2-36b8-4d0d-899a-c72d2aac194b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.403951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:38.430949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.432355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.432825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.433496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.442108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.444144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:38.458198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:38.459186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:38.460085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:38.463009 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 405/798] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:38.470350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.555130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9c560-0fc4-42c5-82c3-9cdd6bd9ac3a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.602136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2a772-d07e-4749-a771-2e098d26ee93 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.604649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.606284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.606635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.606977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.607783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:38.608152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.609701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702ddd8-57fb-49e1-a78d-99de920ef3c7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.639580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 394/799] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:38.639580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca772-7da4-41b5-82e5-afcb03052584 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.639580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca772-7da4-41b5-82e5-afcb03052584 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.639580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca772-7da4-41b5-82e5-afcb03052584 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.641029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 395/800] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:38.641965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ca772-7da4-41b5-82e5-afcb03052584 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.656754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.657755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.658003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.658435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.659176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:38.659545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.660907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea092bd-ce98-44cb-99b3-7f8b0da51496 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.661500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:38.682326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 406/801] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:38.682326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.682326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.682326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.700653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:38.702070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:38.723332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:38.781103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 407/802] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:38.782381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.879850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ca772-7da4-41b5-82e5-afcb03052584 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.891857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc7686-ffea-47d5-b59d-af4d8e0f778e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:38.916523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.917112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.917389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.917659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.928906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:38.929344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.930064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.930532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.931022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.931453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/a3ccebac846245ee915636757eb921f7/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-13dfadb9-f565-4520-8a99-1f324f82943a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:38.947151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:38.948142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:38.949043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:38.949043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:38.949043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:38.950383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=a3ccebac846245ee915636757eb921f7, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:38.952098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23625-4679-44d1-8638-4f9fee63f7a3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:38.953454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 396/803] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:38.954844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bad80-9ff7-4f7b-9dac-62342ff76a56 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.955022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bad80-9ff7-4f7b-9dac-62342ff76a56 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.955303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bad80-9ff7-4f7b-9dac-62342ff76a56 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.960704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 397/804] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:38.960704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bad80-9ff7-4f7b-9dac-62342ff76a56 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:38.974681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 408/805] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:38 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/a3ccebac846245ee915636757eb921f7/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:38.976373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:38.976970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:38.977438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:38.985259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:38.999855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:39.001618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:39.079358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 409/806] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:39.080688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.091383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.187452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bad80-9ff7-4f7b-9dac-62342ff76a56 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.219514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa230e-634a-472d-9a7c-5f14b1bc9bf1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.230913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.232140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.232628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.233240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.234253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:39.234770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.236375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.251324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.254049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 398/807] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:39.258965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.259544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.259888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.260595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.261051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.275827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:39.281734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:39.282758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:39.284070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 410/808] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:39.288611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.380018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a2a2-aa5e-44b4-80af-cbfe7c708f94 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.431624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.438629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.439205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.439480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.439745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.446148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60595e97-0df6-4a1a-be07-5c2a62794419 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.459656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.462713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:39.481573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:39.482498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:39.483459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:39.487573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 399/809] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:39.489300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.510814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.512599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.513416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.513885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.514755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:39.515250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.517594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.529287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 411/810] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:39.531415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.602933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.604796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25252-0687-4313-a255-a0d91d1722f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.643384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.643912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.644190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.644460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.645063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.646405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:39.655891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:39.656867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:39.657726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 400/811] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:39.661985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.664391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c2c8c-9f81-480d-8dad-9c418d77058b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.711561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.712168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.712457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.712738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.713472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:39.713775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.715905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.733000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 412/812] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:39.733000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.745307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.814118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79537b-a810-404c-9a19-b8c54345d964 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.846667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4dc0d-6f24-4a75-a03a-c2c26dbdde24 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:39.863228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:39.868782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.869848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.872276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.873211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.886580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:39.887387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:39.887662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:39.887939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:39.888706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:39.888969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.889768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:39.890337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.891828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:39.900135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 413/813] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:39.901001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:39.910383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:39.911548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:39.912458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:39.912458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:39.912458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:39.914978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 401/814] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:39.920834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.021235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f6b0-ccc1-4024-a2e5-e5fb286e30c1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.058925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.063027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d1d97-b0eb-4163-b851-649999f56069 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.074034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.075748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.076034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.076308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/c4389bd42da64dccbf958503dccf87e1/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.078269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=c4389bd42da64dccbf958503dccf87e1, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.079690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.094448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 414/815] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:39 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/c4389bd42da64dccbf958503dccf87e1/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:40.095248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.108552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.110073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.110572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.111411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.112335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:40.113228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.115836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3092116-cd61-467a-886a-03c130f1ee96 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.124606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 402/816] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 07:11:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:40.126659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071928-21fa-4175-bd90-86adfb754c3b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.126945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071928-21fa-4175-bd90-86adfb754c3b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.127218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071928-21fa-4175-bd90-86adfb754c3b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.133140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 403/817] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:40.133140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071928-21fa-4175-bd90-86adfb754c3b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.156604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.237707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76e95-7aea-4abd-9840-51fd686045d6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.270071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071928-21fa-4175-bd90-86adfb754c3b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.286502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.287965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.288372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.288647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.289329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:40.289833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.291492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16149c92-bb46-4920-8c21-85ab82c58b0e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.308326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 415/818] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:40.310398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.310707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.310955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.311611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.312367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.312792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.313241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.314193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.315675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.331167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:40.332577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:40.349135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 404/819] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:11:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:40.350427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.386678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.414141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 416/820] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:40.512491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bec50-2012-4f71-87cc-f97eff1984b2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.548960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.549523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.549796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.550062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] PATH_INFO: `/v3/users/nova` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.558608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.560849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:40.572868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:40.573976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:40.574995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 405/821] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:11:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:40.581563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.620985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.731933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a256c-cf35-4f73-b043-071d9cee11d4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.770727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:40.771486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:40.771874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:40.772233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:40.773085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:40.773483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:40.775166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:40.789524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.803354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 417/822] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:11:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:40.808635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:40.892429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.904883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:40.923014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2cb6-de77-4016-b245-57fcf37053df None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:40.941533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbde35-e1c7-4842-bded-eec85d244863 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.002634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.003244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.003521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.003791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.004913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.005557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.005828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.006094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] PATH_INFO: `/v3/services/network` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.006675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.009901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:41.025951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:41.026989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 07:11:41.028364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:41.034391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 406/823] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:40 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:41.037031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.114105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.115874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.201010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa26d5-8abe-465c-8e33-1593a68d8595 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.227561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:41.241646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.242780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.242942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.243170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.243828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:41.244115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.245506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.247864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 418/824] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:11:41.252616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 407/825] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:41.253804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.257934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.278408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:41.395526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453671f7-03f8-4cef-9951-3d6f6d5153fe None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.460204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f20d-443d-4868-aa7b-ff07b0e2c4d5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.477617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.478624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.479084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.479531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.501845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.504614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a52f0333-a5ed-4acf-9357-454cb67f1331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:41.532216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:41.533244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:41.534136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:41.534136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:41.534136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:41.536558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.537423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.537843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.538274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.539369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] RBAC: Adding query filter params (type=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:41.539676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.541061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.542665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 419/826] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:41.544166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6bcc-9231-414f-b0f2-85c5340c213a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.544667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6bcc-9231-414f-b0f2-85c5340c213a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.545184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6bcc-9231-414f-b0f2-85c5340c213a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.550801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 408/827] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:41.553424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 420/828] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:41.554352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.555528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6bcc-9231-414f-b0f2-85c5340c213a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.598014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:41.634366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:41.670314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ceb03-eefb-49bb-92e8-8ef004c9665f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.681254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6bcc-9231-414f-b0f2-85c5340c213a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:41.718355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.719105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.719385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.719659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.720331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.721860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff7603-dc21-438e-a825-2e2978a9251b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.735061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:41.735928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.736271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.736647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.737435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:41.737929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:41.739493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:41.752186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 409/829] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:11:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:41.753874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:41.758685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:41.758847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 421/830] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:41.759542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:41.761905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:41.780833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:41.782542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:41.869601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 410/831] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:41.948577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:41.959325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29761f2-35c3-474a-8b76-1fa06b75d9a7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.001161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.001790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.002061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.002324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.009553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.011002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:42.030345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:42.031299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:42.032183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 422/832] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:42.037944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.116453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:42.164912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3b6cd-a97d-4ad1-9976-ce3405ccc41f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.200155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.200772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.201078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.201352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.202051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:42.202354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.203775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.221861 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 411/833] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:42.227321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.236870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.362295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6174-4e2b-4d30-b1db-da7665991d6c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.372551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede0500-054d-49e5-8fcf-ef1262539619 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.402294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:42.409613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.410737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.411228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.411653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.415786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.416461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.416830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.417203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.418016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:42.418398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.420529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.425480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:42.427817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 423/834] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-264cdb92-610b-4ab2-9783-c67e0778a69c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:42.432323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:42.433325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:11:42.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:42.442502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 412/835] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:42.444160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.467355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:42.566447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ca399-47ac-4e43-9fec-eec02b3f8114 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.606376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a8af-df73-4504-8335-64729107f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.607666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a8af-df73-4504-8335-64729107f83e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.608103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a8af-df73-4504-8335-64729107f83e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.608527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a8af-df73-4504-8335-64729107f83e None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.609377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136a8af-df73-4504-8335-64729107f83e None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:42.609832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136a8af-df73-4504-8335-64729107f83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.611268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136a8af-df73-4504-8335-64729107f83e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.620442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 424/836] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:42.626654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044665-609c-4b8b-a4ac-6910b2835ace None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.626928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044665-609c-4b8b-a4ac-6910b2835ace None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.627295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044665-609c-4b8b-a4ac-6910b2835ace None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.628382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a8af-df73-4504-8335-64729107f83e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.628983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 413/837] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:42.633986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.634204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.635126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.656080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:42.657026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:42.728725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:42.752558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a8af-df73-4504-8335-64729107f83e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.753096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 414/838] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:42.795727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.796773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.797301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.797773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.810234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.811982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:42.827125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:42.828077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:42.828972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:42.828972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:42.828972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:42.830972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 425/839] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:42.836282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:42.944816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395201a6-488f-4f69-9368-c05b0e01ef47 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:42.947137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:42.979699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:42.980215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:42.981033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:42.981312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:42.981950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:42.982327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:42.983787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc05fc-09f1-41fd-9bf4-e0902091739f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:42.995929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 415/840] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:43.002000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.007475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07563-f6bb-46f6-a5f0-919951cd43f8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.007756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07563-f6bb-46f6-a5f0-919951cd43f8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.008168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07563-f6bb-46f6-a5f0-919951cd43f8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.009776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 416/841] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:43.018637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.018921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.019182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.051984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:43.053709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:43.123509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5903-54a3-475b-a0f6-cba694ce8c8a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.138013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 417/842] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:43.139233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.161120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.169761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.170505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.170849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.171805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.183780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.185497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:43.201130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:43.202227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 426/843] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:43.207993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.271638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.282589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f2d72-5927-4afc-bb07-a7c21013a10e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.320536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.321559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.322134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.322602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.335505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:43.341505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:43.342504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:11:43.343349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:43.344467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8a72-2faa-4093-9965-1e1db9c9339f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.346185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 418/844] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:43.352651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.392170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.392804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.393128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.393399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.394133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:43.394448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.396422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.426205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 427/845] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:43.427573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.495091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.497809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e030564-ef0e-410a-89a0-7b5c6eafe202 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.542615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.543374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.543668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.544063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.545090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:43.545399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.547268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.550315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c659cc-d789-4e4e-ae3e-efbdb7969c38 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.558303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 419/846] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:43.559906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.580513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.591482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.592215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.592483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.592681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.593299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.594660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:43.603521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:43.604773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:43.606148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 428/847] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:43.608100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.696165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f925d-71e8-4789-a85a-20de1ec0838d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.722666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbeb81-ec9f-45b1-99c4-2044ba54774d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.739351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.740283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.740898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.741204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.741463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.766219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.766855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.767130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.767400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.780220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.781728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:43.796925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:43.798020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:43.799263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:43.799263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:43.799263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:43.799263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 429/848] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:43.799263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.829123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.830558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.849694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:43.918565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522989-7258-49da-9091-a14a6ab77b67 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:43.943505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 420/849] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:11:43.944366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:43.960191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:43.961111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:43.961571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:43.961904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:43.962531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:43.962832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:43.964203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:43.969103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 430/850] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:43.970166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.006687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.061489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cb1b7-2dec-4262-80f9-0f741413b8db None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.079457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.082251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a37d6-0910-414c-adc3-d26f0cceaee0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.099179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.099783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.100060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.100331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.100988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:44.101292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.102640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.114281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 421/851] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:44.116094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.116717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.117019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.117290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.118721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.128599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.130065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:44.145419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:44.146458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:44.147368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:44.147368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:44.147368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:44.147368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 431/852] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:44.147368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.232102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf720cbf-20f0-44d8-a8fe-af8a3e3c50e0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.239744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.263435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5376f9e-987b-4896-836e-63702469b6ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.267752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.268352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.268625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.268890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.269539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:44.269837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.271185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.284967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 422/853] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:44.284967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.315920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.317145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.317559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.317834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.330523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.332156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.333342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:44.348183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:44.349214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:44.350155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 432/854] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:44.355983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.415243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164773-d582-4fe6-99c5-a11d0f9c383f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.464502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.465611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.465884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.466156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.466820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:44.467127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.468714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.475782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564177d-59f5-4660-bb2b-3f70c0f93a82 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.488177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 423/855] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:44.489163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.522281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.522921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.523152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.523417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.524137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:44.524437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.525891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.541177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.546646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 433/856] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:44.551726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.613767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.617674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a250d7d-2782-4a90-a8e5-f12b4a410dde None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.664533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.665322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.666261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.666607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.677126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.678885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.679498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66940-8ad0-4099-bd84-dd0a8ca4a4d1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:44.693398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:44.694432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:44.695403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:44.695862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 424/857] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:44.700999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.735777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.736648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.736923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.737218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.814941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba288a2-8284-4fe7-b34a-7e7a59ce5a00 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:44.823671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.828091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.829928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbb34f-c2c4-44db-bf4d-57e47d174353 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.851223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:44.851841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.852119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.852388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.853104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:44.853410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:44.854827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a7637-591b-4c32-86ed-80315bc3b166 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:44.873093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 425/858] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:44.873093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb51f7-8a8c-4b64-8b45-144b788f8182 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.873093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb51f7-8a8c-4b64-8b45-144b788f8182 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.873093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb51f7-8a8c-4b64-8b45-144b788f8182 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.873479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 426/859] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:44.874379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb51f7-8a8c-4b64-8b45-144b788f8182 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:44.898106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:44.953865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 434/860] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 07:11:44.955083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:44.955373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:44.955650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:44.973691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:44.974873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:44.983266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb51f7-8a8c-4b64-8b45-144b788f8182 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.020356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.020975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.021245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.021516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.022230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=a3ccebac846245ee915636757eb921f7) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.022535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.023933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.036980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 427/861] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:44 2022] GET /identity/v3/role_assignments?user.id=a3ccebac846245ee915636757eb921f7&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:45.036980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.065432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 435/862] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:45.129952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.146605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3a16-24d4-4a13-b13c-67ca1ecc7606 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.182385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.183028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.183304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.183599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.195561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.197032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:45.212655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:45.213719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:45.214611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:45.214611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:45.214611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:45.214611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 428/863] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:45.220599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.258158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.338081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae50b-77a2-4d8a-be88-82f1eb72e8cd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.353056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.377843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.378442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.378786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.379088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.379702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.380061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.381418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.395842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 436/864] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:45.401892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.472812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f8750-aee0-49c0-a2be-fa7d73bbad23 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.506704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.518321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.518951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.519181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d52af-6736-46b9-bef9-afb3c499c63a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.519297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.519477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.520122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.521596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.524283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:45.533561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:45.534769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:45.535968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 429/865] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:45.542804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.567646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.568287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.568563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.568828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.577568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.579026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:45.591384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:45.592509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:45.593661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:45.597031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 437/866] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:45.598330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.665098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736ca0-9133-4b06-bedc-2da2def06960 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.712977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.713599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.713878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.714150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.714811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.716559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.718258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.725889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 430/867] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:45.731130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.731742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25f290-7cb6-454a-8ac7-a82d9c509dbe None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.781069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.781968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.782248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.782521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.783465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.783802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.785611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.796842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.809109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 438/868] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:45.821127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:45.831942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:45.839466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17a1a-3b17-4c76-9c11-724eef429e5a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.874409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.875174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.875536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.875894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.876604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.877028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.878462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91fbca-6ef5-496b-ad70-319234562969 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.889043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 431/869] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:45.938086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db4d8-3415-41a4-801c-5f1a77e74a33 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:45.979674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:45.980685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:45.981154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:45.981572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:45.982536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=c4389bd42da64dccbf958503dccf87e1) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:45.983004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:45.984712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5144-50c9-4a45-8cc0-0e243e5d225f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:45.992197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 439/870] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:45 2022] GET /identity/v3/role_assignments?user.id=c4389bd42da64dccbf958503dccf87e1&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:46.560527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16638323-a92a-498e-92bf-1befa7222332 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:46.561103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16638323-a92a-498e-92bf-1befa7222332 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:46.561103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16638323-a92a-498e-92bf-1befa7222332 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:46.562802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 432/871] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:46.568046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976ef5-3233-41d5-be74-6ee343861e3c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:46.568335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976ef5-3233-41d5-be74-6ee343861e3c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:46.568606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976ef5-3233-41d5-be74-6ee343861e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:46.587288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-84976ef5-3233-41d5-be74-6ee343861e3c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:46.588458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-84976ef5-3233-41d5-be74-6ee343861e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:46.659176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 440/872] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:46.937062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16638323-a92a-498e-92bf-1befa7222332 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:47.041879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16638323-a92a-498e-92bf-1befa7222332 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:47.073100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:47.081790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:47.082397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:47.082671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:47.082939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:47.083595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:47.084930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e75ca-6b90-407a-800b-e50f3b0c7097 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:47.100631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 433/873] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:11:46 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:47.483689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce6b6-587f-4117-96db-f5bcc714b851 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:47.484256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce6b6-587f-4117-96db-f5bcc714b851 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:47.484256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce6b6-587f-4117-96db-f5bcc714b851 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:47.485812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 441/874] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:47.491178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:47.491467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:47.491746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:47.511256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:47.513125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:47.589481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 434/875] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:47.858216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfce6b6-587f-4117-96db-f5bcc714b851 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:47.862179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:47.968366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfce6b6-587f-4117-96db-f5bcc714b851 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.003852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.004502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.004779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.005094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.018559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-25aa9d06-ef17-45e9-ba4a-fdf1e2df3a62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:48.024213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:48.025288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:48.026137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 442/876] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:11:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:48.032212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.135315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016b5d-3b06-4635-bc45-1d0af4adefaf None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.135505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016b5d-3b06-4635-bc45-1d0af4adefaf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.135769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016b5d-3b06-4635-bc45-1d0af4adefaf None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.137583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 443/877] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:48.142504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588ffcf-e351-4e57-b011-2fe0bd5c4f6e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.150767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9c01f-7883-47d0-a474-0987ad73c823 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.151335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9c01f-7883-47d0-a474-0987ad73c823 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.151803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9c01f-7883-47d0-a474-0987ad73c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.160868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.182530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-05a9c01f-7883-47d0-a474-0987ad73c823 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:48.184904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-05a9c01f-7883-47d0-a474-0987ad73c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:48.186416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.187058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.187330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.187592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.188415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:48.188541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.190484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.203859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 435/878] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:48.206506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.253966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.269741 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 444/879] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:48.295127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189daba-ffbf-49a7-979b-8aadbe7f4142 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.295483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189daba-ffbf-49a7-979b-8aadbe7f4142 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.295672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189daba-ffbf-49a7-979b-8aadbe7f4142 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.297365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 445/880] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:48.302637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.302921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.303195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.322496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:48.323698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:48.338052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd45ad-9f63-42ea-a8da-418596b8b594 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.375737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.376796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.377306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.377782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.390856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.392604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:48.409069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:48.410021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:48.410904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:48.410904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:48.410904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:48.411073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 446/881] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:48.412971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 436/882] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:48.424998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.541161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ad6d9-3032-4675-84c4-2133b0b251f5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.558053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.569692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.582238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.583274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.583553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.583824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.584750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:48.585079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.586626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.591354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.605326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 447/883] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:48.609006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.674111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc330a6-bb54-4405-a351-2071e3e873c0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.720123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.720731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.721022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.721294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.721950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.723351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.732980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cfdda-3068-4095-a084-30d61b0609e6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.739922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 437/884] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:11:48 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:48.741246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.786662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.787675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.787955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.788276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.798566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.800116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:48.815185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:48.816212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:48.817535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 448/885] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:48.823016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.859559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.876784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf2cd-ca53-4c46-a78e-1c063674acc6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:48.892406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:48.925719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:48.926274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:48.926544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:48.926809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:48.927396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:48.928754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:48.940084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:48.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:48.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:48.942918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 438/886] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:48.949238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:48.987685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578506b4-b4ba-4bd5-b957-c7c75259f149 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.037860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.038467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.038743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.039008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.040254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:49.040562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.042486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.067514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 449/887] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:49.073616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.074443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe0c37-e421-4019-a2e7-e8ffa8303b9a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.122889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.123648 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.124044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.124322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.125223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:49.125528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.127291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.134379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 439/888] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:49.140274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.167421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.186243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.192834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cd49e-5598-4d92-b3fd-dd1cb92cc16e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.240455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.241577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.242338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.243807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.288668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1aa0-1c91-480a-8a0e-08c5d44d4ca1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.333758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.335577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.335945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.336630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.336993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.337350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.338053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:49.338428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.341659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4521c-936a-4da1-9fdd-291bf6224e50 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.357000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 440/889] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:49.357000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b8827-ce14-40a3-bf0b-fa58b12103ab None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.357000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b8827-ce14-40a3-bf0b-fa58b12103ab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.357000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b8827-ce14-40a3-bf0b-fa58b12103ab None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.357000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 441/890] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:49.360376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.360661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.360937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.383144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:49.384622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:49.476727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 442/891] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:49.477719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.488605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 450/892] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 07:11:49.489923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.504431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.609221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba1fcb-121c-448b-9933-526a85ae2cce None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.649196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.649831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.650106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.650378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.661702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.663206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:49.679559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:49.680521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:49.681697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:49.681697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:49.681697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:49.681697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 443/893] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:49.687239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.757379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.809338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3282-4148-4c3b-a292-4729763dca8f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.836573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.853230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.853845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.854123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.854396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.855047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:49.855380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.857772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.872985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 451/894] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:49.878934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:49.893003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5d9ee-af0e-4aa5-badf-6ba848de7759 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:49.936383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:49.937038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:49.937315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:49.937609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:49.938269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:49.939619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:49.950141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:49.951168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:49.952000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:49.956533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:49.956651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 444/895] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:49.957848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.008630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f397c88-985a-4ae5-803e-9f598dfc65e3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.051705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.052713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.053228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.053728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.062151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.063898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.073787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577834b-8ec7-49c8-99b7-e47a4626d474 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:50.076977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:50.077936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:50.078818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:50.081257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 452/896] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:50.093002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.101044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:50.112530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.113200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.113520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.113789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.114753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:50.115059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.117187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.127307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 445/897] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:50.128852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.209512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e17641-a85d-4fd4-9ccb-067f58bee420 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.227558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:50.246199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.246834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.247107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.247398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.248048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:50.248349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.249798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.252999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625acda7-b7d9-4742-a033-9305ea1ac0d4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.267435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 453/898] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:50.269733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.297391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.298086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.298472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.298808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.299730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:50.299955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.301407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.313036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 446/899] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:50.314370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.379476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:50.392248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c0c85-05eb-4bcb-8407-9796c3bae127 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.429726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.430471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.431098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.431459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.432012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057e849-162c-465b-b62b-c93ed19c6806 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.432374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] RBAC: Adding query filter params (role.id=1d422933778c46bda856e31479a42a9f, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=a3ccebac846245ee915636757eb921f7) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:50.432772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.434465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.444988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 454/900] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:11:50 2022] GET /identity/v3/role_assignments?user.id=a3ccebac846245ee915636757eb921f7&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=1d422933778c46bda856e31479a42a9f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:50.478331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.478944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.479220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.479487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.480190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] RBAC: Adding query filter params (interface=public, service_id=609e41d864874647a0c022020aa0bf79, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:50.480492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.481971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b3799-ee57-4d7e-9537-25a56c5d03fe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.488423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 447/901] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:11:50 2022] GET /identity/v3/endpoints?service_id=609e41d864874647a0c022020aa0bf79&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:50.494170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:50.544347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:50.609235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5bb06-3919-4d81-b862-c8451cd57d85 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:50.647685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:50.648306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:50.648625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:50.648897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:50.649537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:50.651038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34483cb3-abda-46cc-a267-13727cf2741e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:50.657609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 455/902] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:11:50 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:11:51.063918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c11-b75a-4d5d-8582-553e94ecedff None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.064465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c11-b75a-4d5d-8582-553e94ecedff None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.064465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c11-b75a-4d5d-8582-553e94ecedff None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.065943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 448/903] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:51.071417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.072228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.072715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.091464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:51.092631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:51.162695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 456/904] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:51.437789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803c11-b75a-4d5d-8582-553e94ecedff None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:51.542758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803c11-b75a-4d5d-8582-553e94ecedff None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:51.563912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:51.584710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:51.585391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.585667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.585934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.586529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:51.587890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b3bdce4c-5bb0-4523-84e4-dd5843e29909 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:51.597715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:51.598695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:51.600024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 449/905] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 07:11:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:51.607784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:51.643804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:51.685801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed708-82c5-4bcd-bf2a-1ec06ae48c80 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.686113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed708-82c5-4bcd-bf2a-1ec06ae48c80 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.686295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed708-82c5-4bcd-bf2a-1ec06ae48c80 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.687939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 450/906] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:51.694495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.694775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.695050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.713889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52de333-81b3-428d-b718-5d1ab7b6137d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:51.720550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:51.722132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:51.750872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:51.751489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.751756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.752020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.752649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:51.752979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:51.754328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:51.765002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 457/907] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:51.765902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:51.800961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 451/908] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:51.869971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449abb-1556-45d4-aae2-f74602db6b81 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:51.873278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:51.904548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:51.905185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:51.905465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:51.905730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:51.906351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:51.906649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:51.908004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:51.917288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 458/909] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:51.919893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:52.019584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12dd5f-a26c-42bc-bce0-34b26022a9c9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:52.033322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:52.054139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:52.054735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.055003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.055274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.056218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] RBAC: Adding query filter params (interface=public, service_id=eaa8d8833eea47cb8434c8194b3c5fc1, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:52.056523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:52.057908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:52.068236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 452/910] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:11:51 2022] GET /identity/v3/endpoints?service_id=eaa8d8833eea47cb8434c8194b3c5fc1&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:52.069677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:52.108895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:52.174662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1764-5e57-4cc1-9241-363a366bb373 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:52.213566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:52.214283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.214630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.215621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.216324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:52.217821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12861d5b-bff3-4866-82b9-25f35367033f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:52.218491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6ca01-9813-4d96-b5c5-b63e798997a9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:52.229565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 459/911] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:11:52 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:11:52.258282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:52.258907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.259183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.259454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.260123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:52.261651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48357025-025f-4b33-a88f-8756ef7d01d5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:52.279963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 453/912] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:11:52 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:52.769596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265706f-2f22-4979-ab0b-1e2e885c72bb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.770287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265706f-2f22-4979-ab0b-1e2e885c72bb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.770774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265706f-2f22-4979-ab0b-1e2e885c72bb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.772761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 460/913] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:52.779404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7cbdc-ff15-4297-a44e-4a7e66331121 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.779720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7cbdc-ff15-4297-a44e-4a7e66331121 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.779953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7cbdc-ff15-4297-a44e-4a7e66331121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.798895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6bd7cbdc-ff15-4297-a44e-4a7e66331121 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:52.800089 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6bd7cbdc-ff15-4297-a44e-4a7e66331121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:52.831375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:52.878696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe9e7-35fc-4c85-887b-145d43e81d56 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.878924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 454/914] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:52.879023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe9e7-35fc-4c85-887b-145d43e81d56 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.879230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe9e7-35fc-4c85-887b-145d43e81d56 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.880564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 461/915] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:52.885822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:52.886311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:52.886825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:52.905036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:52.906468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:52.973559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 455/916] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:53.160164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbe9e7-35fc-4c85-887b-145d43e81d56 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.190940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.249391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.275018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbe9e7-35fc-4c85-887b-145d43e81d56 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.319179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.320235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.320523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.321079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.335545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:53.342225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:53.343147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:53.343966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:53.348522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 462/917] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:53.350307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.365725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.377620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b8d0-0ee0-42ed-87b1-083a07f7c602 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.420716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.421387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.421664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.421935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.422521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.424391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:53.435394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:53.436380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:53.437666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 456/918] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:53.443276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.479706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.499536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea54216-2473-4503-a4a5-c4df2ac30b02 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.546504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.547254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.547432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.547671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.548335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:53.548635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.550040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.564129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 463/919] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:53.566128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.568983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db6d9-5b7f-4cca-ab95-78ddbbf2db98 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.612068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.613096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.614046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.614563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.616158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:53.616682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.619119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.627394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 457/920] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:53.632575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.661636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.687590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cae078-af6f-45f7-92fe-a082480676aa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.731763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.732503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.732852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.733221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.748394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.750455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.762763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87362-13d6-4905-88b7-1a3416ceab7b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:53.769666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:53.770608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:53.771512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:53.771512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:53.771512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:53.773089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 464/921] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:53.777054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.781012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.804928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.806109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.806610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.807245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.808082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:53.809263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.810986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.821070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 458/922] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:53.822548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.905347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c0546-83f8-43dc-88c0-77403b578deb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.920409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:53.942740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b1daa-e91b-4f2c-b9d4-627b0973eb17 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:53.946866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.947722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.948199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.948706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.950330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:53.950925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.952583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:53.966602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 465/923] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:53.972854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:53.984555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:53.985577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:53.986098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:53.986611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:53.987496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:53.989189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.020546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 459/924] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:11:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:54.047337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:54.093427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa0e7e-c855-4991-8b3e-cfb1815b501f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:54.145551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:54.146184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.146455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.146720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.155303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:54.156764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:54.169898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:54.170944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:54.171910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 466/925] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:54.177241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:54.284143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c236293-2aff-4ca3-a5f9-d764ced5098a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:54.319593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:54.320200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.320469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.320736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.321498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:54.321799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:54.323221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.333907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:54.344832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 460/926] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:54.350252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:54.460355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781846f-45bb-4331-94a8-8ab2b66e9c5e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:54.481441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:54.501465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:54.502443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.502923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.503426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.515245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:54.517001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bbe5c2cb-3be5-4485-989c-85e4c91cb8b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:54.532380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:54.533386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:54.534343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:54.534343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:54.534343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:54.536411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 467/927] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:54.541763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:54.560489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e02e8-a557-48d2-ba9a-b45d87da0ddc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.561153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e02e8-a557-48d2-ba9a-b45d87da0ddc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.561837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e02e8-a557-48d2-ba9a-b45d87da0ddc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.563945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 468/928] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:11:54.570215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927c1af-f40f-4f9f-a6b5-e7e096884b04 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.570779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927c1af-f40f-4f9f-a6b5-e7e096884b04 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.571395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927c1af-f40f-4f9f-a6b5-e7e096884b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.591743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3927c1af-f40f-4f9f-a6b5-e7e096884b04 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:54.592925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3927c1af-f40f-4f9f-a6b5-e7e096884b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:54.650111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e7929-2d74-4ba7-b6bf-1585a12f5a5e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:54.667667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 469/929] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:54.674842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a7ee-9594-4dc5-94bf-d18d46b6ef3e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.675209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a7ee-9594-4dc5-94bf-d18d46b6ef3e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.675555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a7ee-9594-4dc5-94bf-d18d46b6ef3e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.681484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 470/930] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:54.683206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.683591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.683892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.693917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:54.694850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.695326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.695863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.696767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:54.697314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:54.698898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.707457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:54.708650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:54.713496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 461/931] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:54.719578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:54.789151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 471/932] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:54.794622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:54.839568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d6e5e-bc6d-4d15-8413-ae6b7e4507f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:54.874442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:54.875075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:54.875349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:54.875616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:54.882387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:54.883808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:54.895541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:54.896430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:54.897738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 462/933] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:54.903209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:54.934622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:54.960084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.023788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17a868-587c-403e-afaf-36207ff05b3a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.066902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.067863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.068148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.068421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.069430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.069738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.071338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.081592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae82c2d-92b0-4cf1-a263-d7e80073de1f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.083952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 472/934] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:55.084864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.103158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.121745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.122479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.123377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.123377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.123640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.124981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:55.133662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:55.134589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:55.135498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 463/935] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 07:11:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:11:55.137805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.185348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.196388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ea09f-cbc3-43bd-9d2d-8e2fd100a4ac None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.230613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.231259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.231536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.231803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.232379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.233795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:55.242552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:55.243524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:55.244357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 473/936] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:55.245174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.249448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54245f7-8cc4-4caf-a2f6-a579ad9a231a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.285645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.286303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.286564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.286834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/a3ccebac846245ee915636757eb921f7/roles/1d422933778c46bda856e31479a42a9f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.289009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=a3ccebac846245ee915636757eb921f7, role_id=1d422933778c46bda856e31479a42a9f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.290437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.304995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 464/937] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:11:55 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/a3ccebac846245ee915636757eb921f7/roles/1d422933778c46bda856e31479a42a9f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:11:55.309834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.359995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.391084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedac48-33d2-4321-a7d3-4fa8cd3e724b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.432957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55774ef-e552-4ece-b3b1-46b043a8684a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.447292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.448535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.449574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.450819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.451972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.452472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.455085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.462964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 474/938] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:55.469275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.479088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.479706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.480140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.480449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.481212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.481517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.482908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.491725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 465/939] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:55.497345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.498847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.615373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb547-7160-4433-bb30-d39ea2b11e7d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.626831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568e80a-6873-463d-8616-9d9fc684362d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.635506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.662650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.663481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.663783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.664059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.665100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.665420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.667133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.670366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.671219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.671671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.672181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.672871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 475/940] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:55.673196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.673534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.674851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.678621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.684969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 466/941] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:11:55.687512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.802373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:55.822111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d4d5-c3fb-4519-9fd0-976f089e9bc2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.829276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dad7e-bfeb-4624-a6c7-8819b622c1b4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:55.860635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.861289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.861566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.861837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.862711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] RBAC: Adding query filter params (interface=public, service_id=ff7c78a383a2493c9ddcdecb3821bd9d, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:55.863024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.865014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.870786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 467/942] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/endpoints?service_id=ff7c78a383a2493c9ddcdecb3821bd9d&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:55.877276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:55.879139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:55.879754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:55.880029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:55.880300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:55.881160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:55.883300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be702-5cb0-432c-9eaf-d517c58f1a0b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:55.908571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 476/943] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:11:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:11:55.921556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:56.003907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891ea9c-c688-4e1a-9efc-8c40c33fa9e8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:56.050507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:56.051482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:56.051984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:56.052517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:56.053383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:56.054976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7edc82-8c03-4581-91ba-3e220bf9b393 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:56.066813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 468/944] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:11:55 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:11:56.340839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ad8a2-8484-4f99-bb23-33f38b102cab None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:56.341199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ad8a2-8484-4f99-bb23-33f38b102cab None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:56.341423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ad8a2-8484-4f99-bb23-33f38b102cab None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:56.342973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 477/945] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:56.348318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:56.348756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:56.348833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:56.367558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:56.368733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:56.437547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 469/946] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:56.703897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ad8a2-8484-4f99-bb23-33f38b102cab None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:56.812748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ad8a2-8484-4f99-bb23-33f38b102cab None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:56.848880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:56.849540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:56.849813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:56.850082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:56.850833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:56.852312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:56.855055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING keystone.server.flask.common [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 07:11:56.865405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:56.875812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 07:11:56.875812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 07:11:56.875812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [parameters: {'id': '4b881daa17d04f718697f25c1b040222', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 07:11:56.875812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4b881daa17d04f718697f25c1b040222.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4b881daa17d04f718697f25c1b040222. Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:11:56.894832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 28 07:11:56.895704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Sep 28 07:11:56.897329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 07:11:56.898517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 07:11:56.899414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:11:56.900209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'id': '4b881daa17d04f718697f25c1b040222', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:56.903836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:56.906946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4b881daa17d04f718697f25c1b040222. Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:56.909137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 478/947] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 07:11:56 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:56.910413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:56.953386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:57.006081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca7036-38ed-4b32-8b2a-d2c69d9bcbde None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:57.043857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:57.044489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:57.044760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:57.045048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:57.052779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:57.055389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-24e82ac0-9515-447e-bcd5-d6b8e482443a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:11:57.069716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:57.071903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:57.074767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 470/948] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:57.078659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:57.187420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c9f4-50d3-4a28-abe5-565805a4ef27 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:57.226219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:57.226737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:57.227010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:57.227273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:57.227931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:57.228226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:57.229934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eea00-369c-4c0d-ad65-78244a5b2079 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:57.254238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 479/949] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:57.644808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24104d-2dbf-4e41-ba14-a36bc21fbafb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:57.645353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24104d-2dbf-4e41-ba14-a36bc21fbafb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:57.645353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24104d-2dbf-4e41-ba14-a36bc21fbafb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:57.646940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 471/950] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:57.652280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:57.652569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:57.652848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:57.672064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:57.673601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:57.727044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:57.746033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 480/951] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:58.014610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24104d-2dbf-4e41-ba14-a36bc21fbafb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.111401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:58.130459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24104d-2dbf-4e41-ba14-a36bc21fbafb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.165284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.166238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.166710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.167173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.178530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9557bd32-925f-4f71-9f76-a1b83a4b394a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:11:58.183922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:58.185059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:11:58.186052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:58.188625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 472/952] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:58.194366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.213950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453405-e836-4b30-96ba-7b4667e0bcf4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.214261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453405-e836-4b30-96ba-7b4667e0bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.214517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453405-e836-4b30-96ba-7b4667e0bcf4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.217504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 473/953] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:58.222830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.223342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.223609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.244565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:58.245777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:58.309560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec6ef-180a-47ac-9bba-9680f478e27e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.346871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 474/954] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:58.353309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.354336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.354836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.355882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.357120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:58.357662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:58.359351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673b5f-c766-4fdf-99d3-5f964738e10b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:58.368459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 481/955] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:58.374249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.419383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c0fd-7a04-46dd-b286-a37bd190aa8e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.419811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c0fd-7a04-46dd-b286-a37bd190aa8e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.419971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c0fd-7a04-46dd-b286-a37bd190aa8e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.421576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 482/956] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:58.428821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.430146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.430146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.448490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:58.449960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:58.454563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:58.483071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17694ac3-9065-4ba9-b8f9-347c7c961c58 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.498582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:58.522637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.523299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.523578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.523852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.528476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 483/957] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:11:58.537015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:58.538679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:58.553838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:58.554865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:58.555765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:58.555765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:58.555765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:58.558085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 475/958] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:58.563946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.658092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.668764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c05b9-bac8-44de-bdee-4c3a11353a31 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.704389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.705075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.705349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.705663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.706296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:58.706636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:58.708070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:58.724576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 484/959] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:58.730052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.767507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:58.790178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f674d43-ee4a-4d85-8c14-5ef78b3d2886 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.820315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:58.831203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.832018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.832636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.832916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.833673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:58.835037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:58.841313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852cda0-7609-4925-b97f-0875f02b1e31 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:58.844393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:58.845657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:58.846521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 476/960] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:58.847301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.880814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:58.881458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:58.881731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:58.882005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:58.889554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:58.890970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:11:58.902633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:58.903587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 485/961] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:58.904556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:58.974564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7fa8-5f43-43d4-9045-41e00eacd249 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.002636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.009484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbd62-83de-4c99-ac54-0ee4ef1167d5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.010921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.012293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.012563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.012837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.013447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.014800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:59.023866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:59.024888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:11:59.025776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:11:59.029442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 477/962] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:59.030471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.048771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.049418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.049691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.049966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.050584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:59.050882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.052224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.057730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 486/963] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:59.059875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.091788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.146671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78227f5c-f5d0-4b93-aea6-531a00c807ce None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.171904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6b00-ec9f-4960-a2de-df56064c1c4a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.185508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.186118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.186435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.186660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.187391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:59.187689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.189132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.207647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 478/964] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:59.207814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.208167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.208502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.208778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.209429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:59.209729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.211087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.212851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.221270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 487/965] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:59.222861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.256788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.325784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefb8f6-4ad7-4c7e-8414-0f1c9972edad None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.338987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.341131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5287042-caf1-4511-b9cc-5d592dcbe948 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.361535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.362062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.362668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.362668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.363223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:59.363516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.364966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.369529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 479/966] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:59.377306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.387753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.388376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.388651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.388917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.502154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a7ab-b9b9-4396-b7fa-3e9fa6d2ea20 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.505820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.507300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec69087-9635-4c58-9aae-ccd95932416e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.538270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.554829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.555445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.555721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.556243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.556884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:11:59.557212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.559370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af8e0d-211c-4f34-961e-da765b01cd7c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.566478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 480/967] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:11:59.567696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb0ede-5088-4838-a13e-224d63c5e3f0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.568046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb0ede-5088-4838-a13e-224d63c5e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.568321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb0ede-5088-4838-a13e-224d63c5e3f0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.569545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 481/968] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:11:59.574743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb0ede-5088-4838-a13e-224d63c5e3f0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.623974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.628303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 488/969] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:11:59.629935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.629935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.630236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.648816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:11:59.649953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:11:59.687086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb0ede-5088-4838-a13e-224d63c5e3f0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.726768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.727480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.727753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.728018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.728678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.733306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 489/970] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:11:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:11:59.733791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.733874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:11:59.761099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 482/971] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:11:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:11:59.810394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:11:59.859019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855067a2-c711-4d20-90be-3d8d5984db51 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:11:59.910138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:11:59.911079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:11:59.911574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:11:59.912025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:11:59.924469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:11:59.926410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:11:59.943154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:11:59.944179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:11:59.945096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:11:59.945096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:11:59.945096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:11:59.946962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 490/972] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:11:59.956970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.020321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.064665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:00.072173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180cb4e-7ea6-4730-b1a3-dba594f9690f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.112379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.113215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.113583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.113943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.114680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:00.115074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.116522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.130147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 483/973] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:00.137948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.157944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb389-6057-4d48-8aad-a347aa8bf03b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.159688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:00.198574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.199643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.199918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.200181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.200844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.202325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.219482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 07:12:00.219482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 07:12:00.219482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [parameters: {'user_id': 'ff7b7f5710fa497a92f74a5b5ade6140', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 07:12:00.219482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Sep 28 07:12:00.226089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:12:00.226999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 07:12:00.227979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 07:12:00.228882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 07:12:00.229840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 07:12:00.230774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ff7b7f5710fa497a92f74a5b5ade6140', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:00.231627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 28 07:12:00.232436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.233673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 491/974] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 07:12:00 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:00.235266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.306863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade870e-3877-452b-b052-116e6299f4fb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.352245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e73b-b9fc-4480-badd-a4d8e779b0a4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.359571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.360399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.361354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.361800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.377244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.379459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.393727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:12:00.394310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:00.395446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:00.396331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:00.401737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 484/975] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:00.401827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.401827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.401827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.403958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.411757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.414246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.423472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:12:00.434596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:00.435538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 07:12:00.436596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:00.439401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 492/976] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:00.446093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.474226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:00.537507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136824fd-916e-4e71-971c-6379fa2ee15d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.560288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792d8e-76b8-46cd-850b-5191ec3c90ce None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.594359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.595567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.596119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.596631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.598450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:00.599228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.599645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.600343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.600722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.601115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.601913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:00.602304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.602545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.603958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614e4db-685d-4b2a-927f-886e03037da9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.626433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 485/977] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:12:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:00.629409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:00.657286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 493/978] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 07:12:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:00.719954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:00.744302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2693a-b6e1-4905-9b3e-80bd1d45a041 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:00.784792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:00.785435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:00.785710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:00.785979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:00.786712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] RBAC: Adding query filter params (role.id=1d422933778c46bda856e31479a42a9f, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=a3ccebac846245ee915636757eb921f7) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:00.787017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:00.788460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543e7e4-abd3-489b-8fd1-cdf9f56291ad None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:00.796110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 486/979] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:00 2022] GET /identity/v3/role_assignments?user.id=a3ccebac846245ee915636757eb921f7&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=1d422933778c46bda856e31479a42a9f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:01.658421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c228c-5c6d-48fc-be86-f4099fb1595e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:01.658949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c228c-5c6d-48fc-be86-f4099fb1595e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:01.658949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c228c-5c6d-48fc-be86-f4099fb1595e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:01.660344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 494/980] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:01.665476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735e8dd-d79c-45cb-98c9-be52688eaa4a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:01.665767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735e8dd-d79c-45cb-98c9-be52688eaa4a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:01.666044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735e8dd-d79c-45cb-98c9-be52688eaa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:01.685557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5735e8dd-d79c-45cb-98c9-be52688eaa4a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:01.686755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5735e8dd-d79c-45cb-98c9-be52688eaa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:01.763349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 487/981] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:02.031378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c228c-5c6d-48fc-be86-f4099fb1595e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:02.050669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7def-0e31-4865-97a0-18ccb7fdbc6e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.050954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7def-0e31-4865-97a0-18ccb7fdbc6e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.051227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7def-0e31-4865-97a0-18ccb7fdbc6e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.052583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 488/982] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:02.057900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.058186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.058500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.077285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:02.078456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:02.122982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:02.145010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c228c-5c6d-48fc-be86-f4099fb1595e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:02.147090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:02.157102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 489/983] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:02.180629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:02.181580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.181922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.182260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.183004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:02.184479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:02.198812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 495/984] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:12:02 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:02.420044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:02.515980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beae0a-163d-4a05-a4ca-03c8f6e5bd18 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:02.556524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:02.557185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.557461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.557730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.558370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:02.559727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:02.568454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:02.569584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:02.570422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:02.573018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 490/985] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:12:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:02.575916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:02.680736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857e4f-f300-4a97-b887-878a70b44a62 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:02.723927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:02.724530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.724805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.725178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.725804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:02.726104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:02.727466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:02.737574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 496/986] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:12:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:02.739113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:02.799868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:02.841555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402be8c-f1fc-4c3a-9c35-ce6c6397282d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:02.875832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:02.876434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:02.876713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:02.877005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:02.877635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:02.877936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:02.879296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:02.889540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 491/987] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:12:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:02.890763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:02.900107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:02.999820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5ebb-2c08-4116-8f3f-23af8132a08e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:03.034992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:03.035596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.035875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.036145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.036848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] RBAC: Adding query filter params (interface=admin, service_id=ff7c78a383a2493c9ddcdecb3821bd9d, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:03.037188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:03.038548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b707db-e36a-4075-af71-059683b64e37 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:03.048107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 497/988] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Sep 28 07:12:02 2022] GET /identity/v3/endpoints?service_id=ff7c78a383a2493c9ddcdecb3821bd9d&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:03.049295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:03.052751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d89b2e-879c-434a-b5d0-642f12666264 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.053064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d89b2e-879c-434a-b5d0-642f12666264 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.053342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d89b2e-879c-434a-b5d0-642f12666264 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.054655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 498/989] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:03.060012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.060291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.060562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.078954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:03.080437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:03.149331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:03.161349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 499/990] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:03.162944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63677d64-66c0-490d-894a-ed560d7070d3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:03.186998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:03.198050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:03.198651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.199441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.199712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.200305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:03.201685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:03.216977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 492/991] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:12:03 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:12:03.426277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:03.528316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33b4e-e0ba-4a99-957a-879022d07e4f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:03.563536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:03.564165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.564438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.564709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] PATH_INFO: `/v3/services/image` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.565441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:03.566801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:03.575704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:03.576734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:03.577598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:03.581029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 500/992] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 07:12:03 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:03.582979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:03.684378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc58e1-55a6-4be3-b90f-4cfca14eacb8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:03.722800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:03.723398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.723670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.723935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.724555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] RBAC: Adding query filter params (name=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:03.724849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:03.726213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afe02a-fe6c-43e1-a84c-61836d885e8a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:03.731241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 493/993] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:03.736658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:03.801198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:03.842102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788c92b-0a0a-4d47-929d-3cd8052e2206 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:03.875546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:03.876149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:03.876430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:03.876703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:03.877361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] RBAC: Adding query filter params (type=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:03.877667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:03.879029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29aa8-c623-4101-a422-85786da95168 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:03.891925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 501/994] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:04.427127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95fcd-dfc2-45bc-ad7d-394359c39563 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:04.427675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95fcd-dfc2-45bc-ad7d-394359c39563 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:04.427675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95fcd-dfc2-45bc-ad7d-394359c39563 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:04.429183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 494/995] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:04.435002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:04.435285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:04.435561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:04.453809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:04.454984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:04.523533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 502/996] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:04.794076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95fcd-dfc2-45bc-ad7d-394359c39563 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:04.850187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:04.898282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95fcd-dfc2-45bc-ad7d-394359c39563 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:04.931431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:04.932057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:04.932331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:04.932597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:04.933708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:04.935078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:04.943786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:04.944746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:04.946298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:04.949884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 495/997] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:04.951404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.013221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:05.055904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca91ab-6fe0-427b-bf57-86006b3ed478 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:05.091935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:05.092531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.092801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.093112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.093808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:05.094134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:05.095482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:05.104918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 503/998] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:12:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:05.106354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.210976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df229e-3961-49c8-ab17-7d0c9e78b84b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:05.227078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:05.248152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87820172-6084-4107-937c-a68166208635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:05.248773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820172-6084-4107-937c-a68166208635 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.249081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820172-6084-4107-937c-a68166208635 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.249353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820172-6084-4107-937c-a68166208635 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.249999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87820172-6084-4107-937c-a68166208635 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:05.250302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87820172-6084-4107-937c-a68166208635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:05.251712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87820172-6084-4107-937c-a68166208635 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:05.262675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 496/999] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:12:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:05.264074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.365365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a60f-b678-403b-9b85-9299524d2658 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.365557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4052ac-903c-42ac-b082-df64dc906c37 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:05.365669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a60f-b678-403b-9b85-9299524d2658 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.365910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a60f-b678-403b-9b85-9299524d2658 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.367342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 497/1000] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:05.373223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.373497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.373764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.389954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:05.392021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:05.393395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:05.404180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:05.405290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.405785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.406222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.407278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] RBAC: Adding query filter params (interface=public, service_id=acf0c1b905e04cbcae701f6b2a608e2f, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:05.407591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:05.408961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:05.414190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 504/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:12:05 2022] GET /identity/v3/endpoints?service_id=acf0c1b905e04cbcae701f6b2a608e2f&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:05.420603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.468931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 498/1002] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:05.526015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1b63-60a4-4344-9f55-1ccae50baba8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:05.560811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:05.561433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.561707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.561974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.562562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:05.563905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:05.570328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 505/1003] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:12:05 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:12:05.736583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.841051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1b886-4eed-4e03-a5c8-51095fb799c9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:05.851806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:05.877418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:05.878045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:05.878326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:05.878593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:05.879171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:05.880561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:05.890542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:05.891682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:05.892531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 499/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 07:12:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:05.899313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:05.952972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:06.008908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50b2a-3ac4-4943-9c7a-92796181b8fb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:06.044059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:06.044662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.044935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.045227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.045853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:06.046156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:06.047513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57757a97-0208-490e-929b-1e7974a752fe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:06.054203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 506/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:12:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:06.060506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:06.094431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a39ceb-b791-48cb-ac22-0eb3a8572a81 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.094739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a39ceb-b791-48cb-ac22-0eb3a8572a81 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.094999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a39ceb-b791-48cb-ac22-0eb3a8572a81 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.096988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 507/1006] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:06.101491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.101773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.102042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.121260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:06.122971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:06.173972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdccb6-509e-4dd9-92b1-62a70c926aed None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:06.194456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 508/1007] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:06.211342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:06.211949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.212224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.212497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.213159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:06.213463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:06.215421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:06.220531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 500/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 07:12:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:06.226019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:06.273661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:06.330570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b709e-edaf-45d4-9b6c-e1000bfe12d3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:06.364100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:06.364706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.365008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.365287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.365967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:06.367356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc828cf-e101-488d-b9e4-b574513563b3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:06.384175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 509/1009] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:12:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:12:06.465055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:06.506967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:06.575212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7698c-99c3-4afb-9191-63226530131b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:06.616012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:06.616623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:06.616901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:06.617200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:06.617855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:06.619202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f42e-904b-4d35-ad65-340cf8ea5a91 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:06.636984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 501/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:12:06 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:07.748791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec984-d4d1-4528-96ca-a94e5acb171f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:07.749400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec984-d4d1-4528-96ca-a94e5acb171f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:07.749400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec984-d4d1-4528-96ca-a94e5acb171f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:07.750850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 510/1011] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:07.755826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:07.756205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:07.756385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:07.774730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:07.775914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:07.851326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 502/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:07 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:08.124055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ec984-d4d1-4528-96ca-a94e5acb171f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:08.228862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ec984-d4d1-4528-96ca-a94e5acb171f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:08.231444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:08.265299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:08.265929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.266209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.266482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.267068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:08.269275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:08.278591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:08.279604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 07:12:08.280454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:08.284685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 511/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:08.285894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:08.310131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:08.390358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d477-0a3e-4a16-98a2-0d3259b645d9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:08.424546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:08.425297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.425567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.425842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.426473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:08.426770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:08.428130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4712cb-99ad-4dc4-9485-c90313e09d68 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:08.438494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 503/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:12:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:08.439919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:08.546687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da31513-0554-42b1-8b6e-d95694c74322 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:08.569161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883dc19-44df-42df-95b4-4ced117ab3b2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.569439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883dc19-44df-42df-95b4-4ced117ab3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.569711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883dc19-44df-42df-95b4-4ced117ab3b2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.571291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 504/1015] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:08.576117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71477567-fb65-4b3d-90d7-419bab8645a9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.576397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71477567-fb65-4b3d-90d7-419bab8645a9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.576665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71477567-fb65-4b3d-90d7-419bab8645a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.582108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:08.582889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.583171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.583439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.584888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:08.585225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:08.586577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:08.596675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 512/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 07:12:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:08.597640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-71477567-fb65-4b3d-90d7-419bab8645a9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:08.598225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:08.598948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-71477567-fb65-4b3d-90d7-419bab8645a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:08.632975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:08.676594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 505/1017] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:08 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:08.708154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4e89-6fb5-403e-868e-657d7929cc10 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:08.746133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:08.746733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.747007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.747271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.747921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:08.749313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c8106-6a33-4c3c-bebd-7579a7946d3e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:08.777446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 513/1018] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:12:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:12:08.820873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150213-f668-49f2-92ef-b96b833c5fc7 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.821427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150213-f668-49f2-92ef-b96b833c5fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.821552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150213-f668-49f2-92ef-b96b833c5fc7 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.822870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 506/1019] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:08.828907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:08.829245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:08.829519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:08.849439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:08.850645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:08.932389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 514/1020] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:08.952558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5150213-f668-49f2-92ef-b96b833c5fc7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:08.968554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.059476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5150213-f668-49f2-92ef-b96b833c5fc7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.096011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.096642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.096919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.097228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.098333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.099754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.101707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 507/1021] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:09.107979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.124188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.197756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.213431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328db2a6-8f01-4626-a86e-af9ed948c174 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.249764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.250367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.250639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.250905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.251557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.252991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.280224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 515/1022] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:12:09 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:09.294952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.315650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed33c-c3cb-4a48-8e6e-7ee761d2983f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.351384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.352033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.352306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.352580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] PATH_INFO: `/v3/services/image` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.353200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.354612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:09.367045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:09.368130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:09.369045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:09.372080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 508/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 07:12:09 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:09.374655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.446850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.488308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37613e9b-4ab8-4a0d-a041-2c7eeafae545 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.526342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.527484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.527763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.528037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.528664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] RBAC: Adding query filter params (name=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:09.529004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.530376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.536438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 516/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:09.541960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.646586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ad5e5-c1b7-4e50-8b16-56573bedf97b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.680254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.680858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.681413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.681684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.682311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:09.682612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.684048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.691149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 509/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:09.694758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.799578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6903-46dc-4110-a30b-5d7c09fa698a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.830270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.837145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.837745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.838022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.838290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.839041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] RBAC: Adding query filter params (interface=public, service_id=5dfae465289649c0a5bad713e277c429, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:09.839356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.841661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf64f-1081-47bf-baec-0f484b622f28 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:09.851213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 517/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:12:09 2022] GET /identity/v3/endpoints?service_id=5dfae465289649c0a5bad713e277c429&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:09.852267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:09.865493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:09.956355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa272c-5c21-4bd7-a97f-cd979c0037c8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:09.990211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:09.990814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:09.991091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:09.991364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:09.991964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:09.993352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041da13-96a2-4176-a103-11ed4e7365e6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:10.007750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 510/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:12:09 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:12:11.066239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5417a-d6cd-4fc2-b1da-f1b808947af4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.066766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5417a-d6cd-4fc2-b1da-f1b808947af4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.066766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5417a-d6cd-4fc2-b1da-f1b808947af4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.068336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 518/1028] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:11.073967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b582-59ff-4006-aba2-d639ba8de457 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.074471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b582-59ff-4006-aba2-d639ba8de457 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.074732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b582-59ff-4006-aba2-d639ba8de457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.098391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4a25b582-59ff-4006-aba2-d639ba8de457 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:11.100096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4a25b582-59ff-4006-aba2-d639ba8de457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:11.168492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 511/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:11.445050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5417a-d6cd-4fc2-b1da-f1b808947af4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:11.513473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948792-6edd-4060-b094-d799a4b14a1a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.513760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948792-6edd-4060-b094-d799a4b14a1a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.514038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948792-6edd-4060-b094-d799a4b14a1a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.515926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 512/1030] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:11.521495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.521773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.522042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.531612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:11.546980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:11.548758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:11.555212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5417a-d6cd-4fc2-b1da-f1b808947af4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:11.592757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:11.593415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.593692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.593961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.604387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:11.609739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:11.610702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:11.611548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 519/1031] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:12:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:11.616596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:11.631840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 513/1032] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:11.676248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:11.725577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fad86-e0b6-4334-bdc4-ffbf070b8acc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:11.760866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:11.762264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.762558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.762832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.763493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:11.763799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:11.765193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:11.773610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 520/1033] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:12:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:11.779350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:11.879138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa85eb-913c-4fdb-baf9-455d9371d70b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:11.900813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:11.912869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:11.913521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:11.913796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:11.914058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:11.916063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:11.927490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:11.928919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:11.944281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:11.945332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:11.946698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:11.946698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:11.946698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:11.946698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 514/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:11.951010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.016431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67ded-5944-4083-a782-2beb2b41139c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.033825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.054675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.055306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.055586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.055860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.057245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.059660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.060561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb80c-4190-4cba-bab3-05146f317138 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.061452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 521/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:12.067428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.102161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.102777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.103051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.103320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.103959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:12.104258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.106473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.119620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 515/1036] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:12.126064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.180052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43b5c-ae77-4b94-8cdb-732c6c934035 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.182246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.219104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.219620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.219893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.220162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.220797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.222266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9abd8-80eb-4253-8a51-803f7cc297c7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.235690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ecced-81af-42d1-af10-30ed65ec07e3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 522/1037] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:12:12 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9840-e97a-4268-8eb2-52e207aae605 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9840-e97a-4268-8eb2-52e207aae605 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9840-e97a-4268-8eb2-52e207aae605 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 523/1038] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.256976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.257590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.273130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.273776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.274060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.274314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.277600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:12.278853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:12.283920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.285836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:12:12.299208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:12.300329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:12.301347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 516/1039] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:12.306681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.325744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.363484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 524/1040] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:12.442597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfe906-d6e7-4d2b-91b3-2ecf2f6aed0b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.450714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.484932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.485496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.485835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.486123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.486913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:12.487229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.488718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.509027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 517/1041] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:12.515096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.616454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeec3c-ce14-4413-9ec0-6c0471dc4dde None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.640227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.655477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.656084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.656358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.656627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.709758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.743379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.744743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.758630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fa010-3752-4ec4-a04d-1c2748a884ce None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.795136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.795867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.796137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.796400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] PATH_INFO: `/v3/services/image` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.797024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.798351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:12.808483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:12.809502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:12.810343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 518/1042] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:12.815451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.844886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.848544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 525/1043] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:12.854144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:12.922038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b58d1-0f49-46fc-a9dd-7d515793b143 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.927806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:12.958525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:12.959125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:12.959398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:12.959663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:12.960284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] RBAC: Adding query filter params (name=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:12.960580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:12.961966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:12.962291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e8c41-60d7-48eb-a4cb-820d7ff7f88d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:12.967001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 519/1044] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:12.972510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.002431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.002986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.003249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.003514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.014385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.015818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:13.030934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:13.031913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:13.032793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:13.032793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:13.032793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:13.032793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 526/1045] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:13.037612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.071616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:13.082304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc04d-e4ba-4dc0-8332-fa2635c7f805 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.123815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.124467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.124859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.125151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.125775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] RBAC: Adding query filter params (type=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:13.126070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.128037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.138751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 520/1046] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:13.139582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.143586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e0cd-8944-48a6-aaff-4342a4ed005d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.165762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:13.178406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.179016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.179292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.179562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.180201 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:13.180503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.182157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.194687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 527/1047] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:13.200925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.249813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1a6-a062-49c3-a213-13239c6fbc97 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.286308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.286912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.287210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.287486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.288132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.289513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.306671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5af8f-41f8-469f-ace4-ab451a21fea4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.320974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 521/1048] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:12:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:13.351339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.351979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.352256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.352528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.360771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.362259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:12:13.375152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:13.376110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:13.377047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 528/1049] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:13.383079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.400820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:13.500619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3a1e-b820-4cc2-97fc-3bdc9762a330 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.535264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.535876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.536159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.536483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.537152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:13.537462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.538884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada20ec-4b00-41a6-9499-d8dec8bb6350 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.551975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 522/1050] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:13.558858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:13.600204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:13.661851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f999-b87e-45f2-b42c-0f8a1010f982 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:13.702533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:13.703393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:13.703668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:13.703934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:13.704647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] RBAC: Adding query filter params (role.id=1d422933778c46bda856e31479a42a9f, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:13.704970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:13.706386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79536731-523f-421f-a6e4-c5806bf30246 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:13.712286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 529/1051] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:13 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=1d422933778c46bda856e31479a42a9f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:14.510671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6afc-3661-44e0-bd1a-880d698a593a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:14.511330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6afc-3661-44e0-bd1a-880d698a593a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:14.511330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6afc-3661-44e0-bd1a-880d698a593a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:14.512776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 523/1052] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:14.518060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:14.518460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:14.518668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:14.537045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:14.538225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:14.609381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 530/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:14.878260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6afc-3661-44e0-bd1a-880d698a593a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:14.952215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:14.983429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6afc-3661-44e0-bd1a-880d698a593a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:15.016868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:15.017516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.017789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.018053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.028251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:15.032996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:15.033923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:15.034758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:15.037100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 524/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:15.040191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.071981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:15.143846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54ef88-fd55-4cba-bd3f-85d06ae4f69c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:15.177118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:15.177718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.177988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.178253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.178906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:15.179203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:15.180576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:15.188895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 531/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:15.194609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.294466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490c47-cbf9-43f9-99a5-73250877bd16 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:15.323443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:15.330456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:15.331076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.331348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.331614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] PATH_INFO: `/v3/users/b65021698a1849a485385ffb6272b8e6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.350103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] RBAC: Authorizing `identity:get_user(user_id=b65021698a1849a485385ffb6272b8e6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:15.351528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6251c16-f342-4579-9871-61865bdd84f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:15.353347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 525/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/users/b65021698a1849a485385ffb6272b8e6 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:15.359188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.459987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:15.468329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542d2e-47d0-4fbd-b8c9-d7ac740ae012 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:15.502113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:15.502712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.502986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.503252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.529464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5e56-fcc6-4c0d-8190-6303b6eb103a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.529999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5e56-fcc6-4c0d-8190-6303b6eb103a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.529999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5e56-fcc6-4c0d-8190-6303b6eb103a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.531389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 526/1057] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:15.536345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.536622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.536929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.554924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:15.556349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:15.575807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:15.577843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:15.633006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 527/1058] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:15.677832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:15.691534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 532/1059] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:15.693200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.807695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb32a-adb6-42c7-b7a1-f5358b79679d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:15.838800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:15.857221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:15.857856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:15.858136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:15.858408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:15.859528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:15.860974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:15.867532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 528/1060] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:15.868645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.901099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:15.970994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352a15-9dd1-4fc9-ad6d-cb6fbcd4793b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.006095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.006690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.006961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.007226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.007970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=41b0327c241a4cd79399193cf3e207fd, user.id=b65021698a1849a485385ffb6272b8e6) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:16.008276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.009712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e777a3-d598-4b98-b727-162b3c03ba79 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.016992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 533/1061] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/role_assignments?user.id=b65021698a1849a485385ffb6272b8e6&scope.project.id=41b0327c241a4cd79399193cf3e207fd&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:16.016992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524bb94-7a1c-4593-b355-d7dc43ca17a1 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.016992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524bb94-7a1c-4593-b355-d7dc43ca17a1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.017261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d6365-dc84-4867-856d-efe7eddcd471 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.017351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524bb94-7a1c-4593-b355-d7dc43ca17a1 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.018439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 534/1062] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:16.030731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.031426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.031712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.051567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:16.052782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:16.059722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.060254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.060529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.060800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.122427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:16.131529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:16.140816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 535/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:16.145755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.147289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.256995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 529/1064] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 07:12:16.259889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.363893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ec49d-b05e-4480-8952-544dd7031e3a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.398999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.399634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.399910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.400180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.407647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.412690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.414177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:16.428855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:16.435834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:16.437132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:16.437132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:16.437132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:16.437132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 536/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:16.438652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.489095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:16.518040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:16.532222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637094e-736b-49d4-ac23-3f93349a7268 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.561428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38cf9d-4d8e-46bf-8098-2e7f0bd848bc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.569665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.570293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.570572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.570841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.581900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:16.586698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:16.587655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:16.588529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 530/1066] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:16.594308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.600249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.600863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.601179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.601457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.602096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:16.602399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.603745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.616079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 537/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:16.713593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e22d6b-14c6-427d-b246-b54a8442ae5e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.752185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.752799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.753109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.753381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.754049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:16.754350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.755712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.770539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 531/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:16.771567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.878046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:16.883256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c22c75-9624-482a-9414-81869f603b02 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:16.927180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:16.927812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:16.928162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:16.928436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:16.940034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:16.941532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:16.956903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:16.958167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:16.959277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:16.959277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:16.959277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:16.959277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 538/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:16.964866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:16.981097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:17.065414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5145c2-288a-433e-9ee0-7db1f2677e71 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.097093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.097677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.097950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.098215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.098834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:17.099132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.100475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.112013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 532/1070] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:17.117400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:17.217252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379151ea-77d8-4690-8f91-e5db2ed1dedc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.251807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.252427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.252702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.252994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.261784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.263210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:12:17.274063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:17.274982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:17.275944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 539/1071] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:17.280976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:17.348659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:17.379629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77d6b-137b-46cb-8fe5-9853658ccdb5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.412683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.413304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.413583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.413852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.414566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:17.414875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.416276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.433848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 533/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:17.439791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:17.454680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:17.537193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec810b-57b4-4dd6-a58d-3b701d408f96 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.569891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.570514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.570789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.571054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.581510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.582942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:17.597296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:17.598234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:17.599121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:17.599121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:17.599121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:17.599121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 540/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:17.604333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:17.703878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1ae97-4c43-4dba-b4eb-5fa92899f214 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.731877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:17.740114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.740715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.741014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.741285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.741922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:17.742220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.743559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.754893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 534/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:17.760385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:17.822198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:17.859817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eb053-638a-4081-945a-6f55653ff102 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:17.892489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:17.893138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:17.893417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:17.893685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:17.900161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:17.901620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:12:17.913246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:17.914321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:17.915220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 541/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:17.920185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:18.021125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196f868-e3b9-44c1-812e-2ddbec7fc8d9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:18.055164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:18.055787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.056064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.056586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.057328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:18.057631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:18.059156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc253d7-3601-4648-946f-f8e801ef0336 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:18.078156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 535/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:18.079390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:18.173876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349e04-ccb7-4390-acd4-675efdc20c28 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.174222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349e04-ccb7-4390-acd4-675efdc20c28 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.174476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349e04-ccb7-4390-acd4-675efdc20c28 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.175769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 536/1077] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:18.177955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1cd0c-6793-4c35-997e-230dd9bfe7c0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:18.180680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.180970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.181502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.199410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:18.200578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:18.202144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:18.213104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:18.213755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.214031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.214394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/1d422933778c46bda856e31479a42a9f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.217084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=8e403816a5674d45b3cefdc34cffcf7d, role_id=1d422933778c46bda856e31479a42a9f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:18.218503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:18.231224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 542/1078] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:12:18 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/8e403816a5674d45b3cefdc34cffcf7d/roles/1d422933778c46bda856e31479a42a9f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:12:18.248214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:18.273920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 537/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:18.538608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:18.639777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160f85a-256e-4316-a928-00c256af4e9a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:18.673325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:18.673953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.674231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.674500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.685980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:18.690852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:18.691808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:18.692646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 543/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:18.699550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:18.798096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aecdc2-0c3b-498b-a735-ded8d7021d7d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:18.831846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:18.832449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.832725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.833027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.833687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:18.833988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:18.835343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:18.844275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 538/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:18.849972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:18.896011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:18.951589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c5230-de3d-4c62-9f3f-1d7d4c07af48 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:18.984896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:18.985544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:18.985822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:18.986094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] PATH_INFO: `/v3/users/b65021698a1849a485385ffb6272b8e6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:18.987211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] RBAC: Authorizing `identity:get_user(user_id=b65021698a1849a485385ffb6272b8e6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:18.988636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:18.990237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 544/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:18 2022] GET /identity/v3/users/b65021698a1849a485385ffb6272b8e6 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:18.996623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:19.038566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:19.099791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccd288-c837-4c17-82a9-a19f6882839a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:19.133521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:19.134142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:19.134418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:19.134688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:19.135795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:19.138091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbcee-75be-4576-81d3-2faf4d1eb231 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:19.139862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 539/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:18 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:19.145671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:19.240668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142f4-9cf3-4715-88e3-47ec9ae7fec5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:19.253355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:19.274817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:19.275349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:19.275622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:19.275889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd/users/b65021698a1849a485385ffb6272b8e6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:19.277800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] RBAC: Authorizing `identity:create_grant(project_id=41b0327c241a4cd79399193cf3e207fd, user_id=b65021698a1849a485385ffb6272b8e6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:19.279196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f46d84-0963-457a-be54-4dbc3fd6d90a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:19.291473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 545/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:12:19 2022] PUT /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd/users/b65021698a1849a485385ffb6272b8e6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:12:20.365777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f1364-e097-4ee6-903f-aa590ea5c295 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:20.366361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f1364-e097-4ee6-903f-aa590ea5c295 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:20.366361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f1364-e097-4ee6-903f-aa590ea5c295 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:20.367773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 540/1085] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:20.373477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:20.373874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:20.374137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:20.392541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:20.393727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:20.459339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 546/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:20.722464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f1364-e097-4ee6-903f-aa590ea5c295 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:20.824597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f1364-e097-4ee6-903f-aa590ea5c295 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:20.833313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:20.857155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:20.857769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:20.858039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:20.858300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:20.868304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:20.873751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:20.874692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:20.875720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 541/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 07:12:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:20.880770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:20.976982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f82a-68c9-4dd1-a968-465561c3ca68 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:20.978408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.013137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.013835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.014105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.014370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.015023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:21.015322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.016673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.029204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 547/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:12:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:21.030491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.138248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d918-22ee-4af2-8025-bef797a5f46e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.171054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.171678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.171952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.172220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.182537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.183993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:21.197555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:21.199231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:21.200156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:21.200156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:21.200156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:21.200156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 542/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:21.204897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.307826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b647b54-eb61-4038-b58a-cd217fb6b596 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.343408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.344005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.344281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.344548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.345213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:21.345517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.346882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066359-c772-49b8-b2fd-e14f4b945d98 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.358846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 548/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:21.364497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.388222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.406140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13527fa-a886-4620-86f6-af9097a362f8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.406710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13527fa-a886-4620-86f6-af9097a362f8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.406710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13527fa-a886-4620-86f6-af9097a362f8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.408040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 549/1091] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:12:21.413135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.413419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.413694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.431765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:21.433493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:21.452062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.469302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62435e6d-fd5a-4ffa-bcec-2cc538048de4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.502057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.502677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.502808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 550/1092] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:21.502957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.503211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.510720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.512805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:12:21.523762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:21.524695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:21.525671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 543/1093] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:21.530713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.625805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b90f1-6c61-4dfa-8171-4cb8f7885920 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.659322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.659915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.660191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.660461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.661207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:21.661510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.662921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.679576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 551/1094] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:21.685804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.768296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.786385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.788147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050456-761c-4483-85b1-ff82eb7dbf6c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.801120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.823148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.824435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.824964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.825275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.890794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01dd81-884b-47a3-885e-d015800caf6c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:21.893462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:21.894893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:21.925724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:21.926348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:21.926629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:21.926898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:21.938080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:21.942924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:21.943862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:21.944705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:21.945375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 552/1095] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:21.950819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:21.972241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:21.992486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 544/1096] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:21.998900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.021576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.051291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e10-1552-42cb-9492-36baf97d29cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.086492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.087081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.087357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.087625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.088279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:22.088579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.090169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.094088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735e9a-d2bd-4052-b95f-8c55db3ec5ea None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.098232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 553/1097] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.104068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.127286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.127901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.128171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.128435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.138446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.139856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:22.154544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:22.155496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:12:22.156400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:22.156400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:12:22.156400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:22.156400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 545/1098] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:22.161174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.192543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.202985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2e2-c20d-4919-b776-2f553dbdbe58 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.237155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.237783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.238061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.238330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] PATH_INFO: `/v3/users/b65021698a1849a485385ffb6272b8e6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.239473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] RBAC: Authorizing `identity:get_user(user_id=b65021698a1849a485385ffb6272b8e6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.240890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.242508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 554/1099] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/users/b65021698a1849a485385ffb6272b8e6 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.248187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.250789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.262092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc9f64-cd0a-4ee2-bd03-aced8739372f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.296271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.296870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.297166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.297436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.298067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:22.298368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.299720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.311782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 546/1100] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.317309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.348033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a0756-86c8-4a60-976e-f861732e5149 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.381441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.382036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.382315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.382583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.408264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.417477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224b70-a96d-4f10-9a11-74d619e91978 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.451150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.451775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.452053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.452301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.452735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.453805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.455621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.459559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.460997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:12:22.472689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:22.473659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:22.474772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 547/1101] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:22.479605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.557330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 555/1102] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 07:12:22.558771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.582461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cdef-bebc-437b-9497-e650732cc5b9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.615754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.616353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.616627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.616895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.617613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:22.617917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.619329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.631163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 548/1103] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.637801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.651397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.658476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7664-24d9-45db-a362-69050bc6919d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.659920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:22.696364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.697150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.697424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.697696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.699673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.701733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.703144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 556/1104] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.708980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:22.740392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b74de3-84e9-414d-8275-d8062bb6e068 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.774227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.774820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.775093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.775360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.776071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] RBAC: Adding query filter params (role.id=1d422933778c46bda856e31479a42a9f, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=8e403816a5674d45b3cefdc34cffcf7d) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:22.776378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.777982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb1a40-41b0-4a1c-8a52-0b20054cab46 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.783541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 549/1105] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/role_assignments?user.id=8e403816a5674d45b3cefdc34cffcf7d&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=1d422933778c46bda856e31479a42a9f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:22.810873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be4dc-7cec-440b-ad23-722fa395d3de None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:22.848601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:22.849261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:22.849543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:22.849816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:22.850557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=41b0327c241a4cd79399193cf3e207fd, user.id=b65021698a1849a485385ffb6272b8e6) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:22.850867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:22.852305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac4faa-1974-4df2-a6d1-1347718d7d93 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:22.859396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 557/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:22 2022] GET /identity/v3/role_assignments?user.id=b65021698a1849a485385ffb6272b8e6&scope.project.id=41b0327c241a4cd79399193cf3e207fd&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:25.053561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81698b5-b40a-42c9-83c5-a347a7258fcd None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:25.054080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81698b5-b40a-42c9-83c5-a347a7258fcd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:25.054188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81698b5-b40a-42c9-83c5-a347a7258fcd None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:25.055421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 550/1107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:25.061355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:25.061732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:25.061915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:25.080211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:25.081419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:25.106710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:25.146323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 558/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:25.415095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81698b5-b40a-42c9-83c5-a347a7258fcd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:25.491166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:25.514746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81698b5-b40a-42c9-83c5-a347a7258fcd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:25.547101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:25.547710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:25.547982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:25.548246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:25.549370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:25.550773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757a1a0-4f0b-4b69-aaf5-b9e6daa2ce83 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:25.552300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 551/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:25.558787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:25.652536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4113-956a-4b79-935b-e606535af8c9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:25.685754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:25.686282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:25.686553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:25.686834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:25.687450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:25.688872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3b997-367b-499d-b5c1-b03aee22b088 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:25.709860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 559/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:12:25 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:27.877226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd5b3-7843-4450-bbac-ec9c72665a22 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:27.877777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd5b3-7843-4450-bbac-ec9c72665a22 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:27.877777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd5b3-7843-4450-bbac-ec9c72665a22 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:27.879071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 552/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:27.884327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:27.884732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:27.884860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:27.902560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:27.903719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:27.917105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:27.969661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 560/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:28.240070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cd5b3-7843-4450-bbac-ec9c72665a22 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:28.344270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cd5b3-7843-4450-bbac-ec9c72665a22 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:28.378717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:28.379365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:28.379660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:28.379950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] PATH_INFO: `/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:28.381419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] RBAC: Authorizing `identity:get_role(role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:28.382804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:28.384197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 553/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:28 2022] GET /identity/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:28.390631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:28.485655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bc74-fe44-4d5d-996b-3c8477b82bfb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:28.519011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:28.519554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:28.519824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:28.520089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] PATH_INFO: `/v3/users/fa00ece17c5144458011a362472e8093` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:28.534048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] RBAC: Authorizing `identity:get_user(user_id=fa00ece17c5144458011a362472e8093)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:28.535470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:28.537255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 561/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:28 2022] GET /identity/v3/users/fa00ece17c5144458011a362472e8093 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:28.542792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:28.557924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:28.645276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe7ab-da82-4848-a9c6-9ea026a7c907 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:28.678175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:28.678768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:28.679048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:28.679320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:28.744355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:28.746640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:28.760097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:28.842564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 554/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:28.848526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:28.859122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:28.946505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e06d5-c5dc-4ba7-93b8-2522ad31cd4e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:28.979186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:28.979824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:28.980099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:28.980364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:28.981465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:28.982884 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c505-6425-47bd-bd65-ddd191783977 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:28.984470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 562/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:28 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:28.990669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:29.084710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873443-2576-4503-bbf6-3ce85f3c7c81 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:29.097209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:29.117593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:29.118177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:29.118449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:29.118714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:29.119422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=41b0327c241a4cd79399193cf3e207fd, user.id=fa00ece17c5144458011a362472e8093) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:29.119726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:29.121865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17457dad-17b0-47e4-a968-021f6d47d65a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:29.126932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 555/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:28 2022] GET /identity/v3/role_assignments?user.id=fa00ece17c5144458011a362472e8093&scope.project.id=41b0327c241a4cd79399193cf3e207fd&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:31.219002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762a82-aac0-412f-ab96-9716eeadb8d4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:31.219563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762a82-aac0-412f-ab96-9716eeadb8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:31.219563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762a82-aac0-412f-ab96-9716eeadb8d4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:31.220878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 563/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:31.226028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:31.226431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:31.226566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:31.244648 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:31.245888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:31.311439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 556/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:31.596701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb762a82-aac0-412f-ab96-9716eeadb8d4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:31.685830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:31.713607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb762a82-aac0-412f-ab96-9716eeadb8d4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:31.746514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:31.747119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:31.747389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:31.747653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] PATH_INFO: `/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:31.749051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] RBAC: Authorizing `identity:get_role(role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:31.750437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:31.751730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 564/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:31 2022] GET /identity/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:31.757584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:31.852643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab124285-771b-4898-b7b7-8c477c2f8dec None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:31.860253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:31.885798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:31.886639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:31.886915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:31.887182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] PATH_INFO: `/v3/users/fa00ece17c5144458011a362472e8093` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:31.888278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] RBAC: Authorizing `identity:get_user(user_id=fa00ece17c5144458011a362472e8093)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:31.889713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:31.891245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 557/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:31 2022] GET /identity/v3/users/fa00ece17c5144458011a362472e8093 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:31.896886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:31.994972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94293-b294-4aa3-8266-f296f58db12e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:32.027678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:32.028290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:32.028567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:32.028844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:32.029974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:32.031404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776eee5-69b4-4307-a2f4-255ca178ad42 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:32.033066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 565/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:31 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:32.038963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:32.134163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4b11-bfd2-4839-9ccd-8b3aa5c90112 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:32.166617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:32.167271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:32.167542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:32.167808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd/users/fa00ece17c5144458011a362472e8093/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:32.169925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] RBAC: Authorizing `identity:create_grant(project_id=41b0327c241a4cd79399193cf3e207fd, user_id=fa00ece17c5144458011a362472e8093, role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:32.171318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991cc82-2b94-4e3b-a2da-9847be43bb14 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:32.183609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 558/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:12:32 2022] PUT /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd/users/fa00ece17c5144458011a362472e8093/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:12:34.297588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66690472-6125-48f3-bdbf-88f58049212d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:34.298123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66690472-6125-48f3-bdbf-88f58049212d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:34.298123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66690472-6125-48f3-bdbf-88f58049212d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:34.299442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 566/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:34.304701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:34.305181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:34.305287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:34.324485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:34.325724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:34.351789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:34.393133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 559/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:34.664353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66690472-6125-48f3-bdbf-88f58049212d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:34.679499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:34.765004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66690472-6125-48f3-bdbf-88f58049212d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:34.796784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:34.797438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:34.797711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:34.797977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] PATH_INFO: `/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:34.799328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] RBAC: Authorizing `identity:get_role(role_id=10fc5ff7ee5b4af58ecf83b5c5f8bab5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:34.800677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:34.801997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 567/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:34 2022] GET /identity/v3/roles/10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:34.808540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:34.904071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590cbd-4118-4aec-8238-44c026f58df7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:34.937105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:34.937655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:34.937930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:34.938198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] PATH_INFO: `/v3/users/fa00ece17c5144458011a362472e8093` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:34.939296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] RBAC: Authorizing `identity:get_user(user_id=fa00ece17c5144458011a362472e8093)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:34.940717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:34.942295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 560/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:34 2022] GET /identity/v3/users/fa00ece17c5144458011a362472e8093 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:34.947969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:35.026679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:35.045068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0979-14ef-4d75-9c20-c25597381b7c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:35.078036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:35.078623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:35.078895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:35.079162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:35.142899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:35.144324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:35.218669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:35.238149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 568/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:35.243991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:35.274237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:35.340250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa312997-de18-4be6-a15a-3f87e6dfcb18 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:35.373013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:35.373622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:35.373897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:35.374167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] PATH_INFO: `/v3/projects/41b0327c241a4cd79399193cf3e207fd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:35.375269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] RBAC: Authorizing `identity:get_project(project_id=41b0327c241a4cd79399193cf3e207fd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:35.376677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c406b2-5da1-4d8c-801c-d44ab58a36c5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:35.378284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 561/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:35 2022] GET /identity/v3/projects/41b0327c241a4cd79399193cf3e207fd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:35.384305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:35.485087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76af03f-abff-4aee-b813-cb8646e72fc1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:35.517610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:35.518193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:35.518466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:35.518730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:35.519434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] RBAC: Adding query filter params (role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5, scope.project.id=41b0327c241a4cd79399193cf3e207fd, user.id=fa00ece17c5144458011a362472e8093) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:35.519734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:35.521164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af346e39-47dd-4d2b-9b58-e0b226ce124a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:35.527222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 569/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:35 2022] GET /identity/v3/role_assignments?user.id=fa00ece17c5144458011a362472e8093&scope.project.id=41b0327c241a4cd79399193cf3e207fd&role.id=10fc5ff7ee5b4af58ecf83b5c5f8bab5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:37.704520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c7b9-60dd-4c94-bc0c-17aa0f863239 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:37.704972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c7b9-60dd-4c94-bc0c-17aa0f863239 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:37.705097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c7b9-60dd-4c94-bc0c-17aa0f863239 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:37.706452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 562/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:37.711574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:37.711961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:37.712125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:37.730363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:37.731536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:37.770372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:37.797104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 570/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:38.069983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c7b9-60dd-4c94-bc0c-17aa0f863239 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:38.165405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:38.169476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c7b9-60dd-4c94-bc0c-17aa0f863239 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:38.202645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:38.203260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:38.203535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:38.203805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:38.204908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:38.206350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295b06c-5f0f-46bc-8816-879b5f5191c1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:38.207907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 563/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:38.214010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:38.310021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca0b03-fe05-4d3f-aa0b-a955f17bcc13 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:38.343519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:38.344049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:38.344323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:38.344593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:38.345257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:38.346670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9036631-e682-4d60-b24a-ea16ceb92b56 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:38.373784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 571/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:12:38 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:40.546153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed81e1-3c33-42c1-9bd8-e015de20d81d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:40.546694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed81e1-3c33-42c1-9bd8-e015de20d81d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:40.546694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed81e1-3c33-42c1-9bd8-e015de20d81d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:40.547997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 564/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:40.553171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:40.553565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:40.553724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:40.572263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:40.573566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:40.593195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:40.638925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 572/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:40.914863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed81e1-3c33-42c1-9bd8-e015de20d81d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:41.012125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed81e1-3c33-42c1-9bd8-e015de20d81d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:41.044896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:41.045529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:41.045801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:41.046066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:41.047154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:41.048566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf67a08-a432-4bb7-ae7e-679c0d85ed88 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:41.050104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 565/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:41.055804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:41.150440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686864-c6f8-460c-9624-ddb8e2ac1d7e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:41.184027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:41.184551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:41.184826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:41.185120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:41.185774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:41.187184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f683b2-c530-499c-8c19-68a2d1974c5c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:41.208469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 573/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:12:41 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:43.354211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c12d4-a712-4ad7-be6e-5081c81d9f3a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:43.355353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c12d4-a712-4ad7-be6e-5081c81d9f3a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:43.355493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c12d4-a712-4ad7-be6e-5081c81d9f3a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:43.356828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 566/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:43.362175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:43.362592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:43.362723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:43.380320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:43.381509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:43.389287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:43.447075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 574/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:43.715746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c12d4-a712-4ad7-be6e-5081c81d9f3a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:43.750668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:43.817855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c12d4-a712-4ad7-be6e-5081c81d9f3a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:43.850096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:43.850701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:43.850971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:43.851235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:43.860874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:43.865597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:43.866669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:43.867872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 567/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:12:43.873288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:43.967382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b782a7-f4df-4665-a6ae-5b07c8c9b70b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:44.000256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:44.000848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:44.001148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:44.001419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:44.002079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:44.002379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:44.003817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:44.013323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 575/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:44.019758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:44.114020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:44.120611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516fad5-722d-4cd6-945e-4410beba9524 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:44.155501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:44.156122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:44.156402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:44.156673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] PATH_INFO: `/v3/users/85c66780e31046bdbdcdf8fab975f566` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:44.170589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] RBAC: Authorizing `identity:get_user(user_id=85c66780e31046bdbdcdf8fab975f566)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:44.172026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:44.173718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 568/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:44 2022] GET /identity/v3/users/85c66780e31046bdbdcdf8fab975f566 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:44.179586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:44.184333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:44.296598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6209da-d3f0-4fcd-9e53-6084d4ac6c51 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:44.334509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:44.335263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:44.335575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:44.335871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:44.411241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:44.412758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:44.415640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:44.530964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 576/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:44.538358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:44.635710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39fa04-5273-461d-8415-ff595267c313 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:44.670216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:44.670837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:44.671108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:44.671376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] PATH_INFO: `/v3/projects/eeabb1e9cfdd42e9913af98584cc677a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:44.672467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] RBAC: Authorizing `identity:get_project(project_id=eeabb1e9cfdd42e9913af98584cc677a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:44.673915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afda419-3f5c-40ad-b61d-b57eb5529e7c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:44.675538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 569/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:44 2022] GET /identity/v3/projects/eeabb1e9cfdd42e9913af98584cc677a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:44.682130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:44.783245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dc5d4-7e2b-4618-b03b-04c3f8a07838 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:44.785555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:44.818695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:44.819314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:44.819588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:44.819856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:44.820658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=eeabb1e9cfdd42e9913af98584cc677a, user.id=85c66780e31046bdbdcdf8fab975f566) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:44.820976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:44.822411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfbd76-a7fc-41d1-a2ef-abe37b887749 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:44.827834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 577/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:44 2022] GET /identity/v3/role_assignments?user.id=85c66780e31046bdbdcdf8fab975f566&scope.project.id=eeabb1e9cfdd42e9913af98584cc677a&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:46.952108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976feb94-fcfa-4b6f-8041-b06b0f3880a0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:46.952663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976feb94-fcfa-4b6f-8041-b06b0f3880a0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:46.952663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976feb94-fcfa-4b6f-8041-b06b0f3880a0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:46.953911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 570/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:46.959050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:46.959479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:46.959578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:46.977930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:46.979089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:47.044167 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 578/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:12:47.312921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976feb94-fcfa-4b6f-8041-b06b0f3880a0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:47.336785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:47.414568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976feb94-fcfa-4b6f-8041-b06b0f3880a0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:47.448754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:47.449488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:47.449764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:47.450031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:47.459601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:47.464931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:47.466117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:47.467060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 571/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:47.472053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:47.562431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:47.566494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abc19a-6584-4c48-b55d-9a27cadc2588 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:47.599393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:47.599979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:47.600255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:47.600526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:47.601202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:47.601505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:47.602855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:47.610677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 579/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:47.616526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:47.696088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:47.714756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b935-2987-4f17-aac7-4b42fb8d494b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:47.747193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:47.747803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:47.748077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:47.748344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] PATH_INFO: `/v3/users/85c66780e31046bdbdcdf8fab975f566` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:47.749465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] RBAC: Authorizing `identity:get_user(user_id=85c66780e31046bdbdcdf8fab975f566)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:47.750886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:47.752442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 572/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:47 2022] GET /identity/v3/users/85c66780e31046bdbdcdf8fab975f566 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:47.758301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:47.853826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a4115-8001-440a-9e6a-d472903589d8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:47.885649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:47.886815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:47.887080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:47.887339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] PATH_INFO: `/v3/projects/eeabb1e9cfdd42e9913af98584cc677a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:47.888425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] RBAC: Authorizing `identity:get_project(project_id=eeabb1e9cfdd42e9913af98584cc677a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:47.889858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f218f82-da82-4d23-b931-cced4bf6d2bc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:47.891418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 580/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:47 2022] GET /identity/v3/projects/eeabb1e9cfdd42e9913af98584cc677a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:47.897363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:47.993057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0267d-1cbd-4072-a67d-89032bb408c7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:48.025616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:48.026250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:48.026561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:48.026829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] PATH_INFO: `/v3/projects/eeabb1e9cfdd42e9913af98584cc677a/users/85c66780e31046bdbdcdf8fab975f566/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:48.028709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeabb1e9cfdd42e9913af98584cc677a, user_id=85c66780e31046bdbdcdf8fab975f566, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:48.030149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d6b1-b500-4a16-b4da-af9fdca37c23 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:48.043424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 573/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:12:47 2022] PUT /identity/v3/projects/eeabb1e9cfdd42e9913af98584cc677a/users/85c66780e31046bdbdcdf8fab975f566/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:12:50.166859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303fa38-d750-4d69-9ec2-1377722cd4bd None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:50.167506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303fa38-d750-4d69-9ec2-1377722cd4bd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:50.167506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303fa38-d750-4d69-9ec2-1377722cd4bd None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:50.168734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 581/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:50.173887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:50.174292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:50.174432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:50.194530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:50.195727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:50.199193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:50.262052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 574/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:50.549943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303fa38-d750-4d69-9ec2-1377722cd4bd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:50.578779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:50.649662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303fa38-d750-4d69-9ec2-1377722cd4bd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:50.682024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:50.683172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:50.683438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:50.683699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:50.693587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:12:50.698886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:50.699899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:12:50.701084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 582/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:12:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:50.705984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:50.805434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30fef8-fa1f-431f-9812-dab1a8abf8ed None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:50.837810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:50.838399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:50.838676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:50.839491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:50.840126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:50.840425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:50.841803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:50.849486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 575/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:12:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:50.856044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:50.940163 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:50.952305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b14651-15d0-4603-b65d-91fab1a1cf8c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:50.984764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:50.985393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:50.985669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:50.985936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] PATH_INFO: `/v3/users/85c66780e31046bdbdcdf8fab975f566` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:50.987028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] RBAC: Authorizing `identity:get_user(user_id=85c66780e31046bdbdcdf8fab975f566)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:50.988432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:50.989990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 583/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:12:50 2022] GET /identity/v3/users/85c66780e31046bdbdcdf8fab975f566 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:12:50.995945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:50.998751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:51.094036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667291-00a7-4735-ab57-f695a046fddf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:51.126483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:51.127069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:51.127345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:51.127614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:51.191656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:51.193097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:51.194866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:51.287000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 576/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:51.293883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:51.390772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610b4b6-3092-4522-b627-ff347937728c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:51.423277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:51.423883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:51.424155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:51.424422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] PATH_INFO: `/v3/projects/eeabb1e9cfdd42e9913af98584cc677a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:51.425525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] RBAC: Authorizing `identity:get_project(project_id=eeabb1e9cfdd42e9913af98584cc677a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:51.426930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd69e22-8b37-4887-bcf5-b0ee109aab01 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:51.428507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 584/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:12:51 2022] GET /identity/v3/projects/eeabb1e9cfdd42e9913af98584cc677a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:51.434995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:51.532683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fe2c5-fb3c-4742-8495-7c77ad4de8e4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:51.534180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:51.566646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:51.567231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:51.567506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:51.567775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:51.568485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=eeabb1e9cfdd42e9913af98584cc677a, user.id=85c66780e31046bdbdcdf8fab975f566) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:51.568791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:51.570234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee298c-4f21-4697-a45d-5a25f6432e7e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:51.575765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 577/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:12:51 2022] GET /identity/v3/role_assignments?user.id=85c66780e31046bdbdcdf8fab975f566&scope.project.id=eeabb1e9cfdd42e9913af98584cc677a&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:53.707248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c336-452b-4b2b-aff0-37e299650295 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:53.707699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c336-452b-4b2b-aff0-37e299650295 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:53.707803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c336-452b-4b2b-aff0-37e299650295 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:53.709150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 585/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:53.715032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:53.715449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:53.715572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:53.734069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:53.735242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:53.799174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 578/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:54.069687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c336-452b-4b2b-aff0-37e299650295 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:54.094591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:54.169052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c336-452b-4b2b-aff0-37e299650295 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:54.201689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:54.202277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:54.202551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:54.202816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:54.267076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:54.268510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:54.293217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:54.366532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 586/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:12:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:12:54.372779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:54.461665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:54.468223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64295264-3ab7-4070-a4db-437eefbcdf35 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:54.501031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:54.501646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:54.501939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:54.502205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:54.508596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:54.510038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-89a77d3a-baf5-4dbc-a90f-c7bd1b66fcb6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:12:54.521083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:12:54.522285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:12:54.523356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 579/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:12:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:12:54.528912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:54.624639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd2d68-573c-496a-8413-0f22d6d96b1f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:54.647836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:54.658950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:54.659547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:54.659823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:54.660091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:54.660721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:12:54.661047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:54.662408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6013-1820-49c7-82f4-dc56538f8eda None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:54.669394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 587/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 07:12:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:56.773081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3b23-9f4a-4be8-bb8f-3262eeb422d6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:56.773589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3b23-9f4a-4be8-bb8f-3262eeb422d6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:56.773589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3b23-9f4a-4be8-bb8f-3262eeb422d6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:56.774950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 580/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:56.780253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c343-213f-4fa1-ae2a-78a20ef514b9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:56.780667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c343-213f-4fa1-ae2a-78a20ef514b9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:56.780785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c343-213f-4fa1-ae2a-78a20ef514b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:56.798718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-23f6c343-213f-4fa1-ae2a-78a20ef514b9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:56.799883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-23f6c343-213f-4fa1-ae2a-78a20ef514b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:56.865474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 588/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:57.133997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc3b23-9f4a-4be8-bb8f-3262eeb422d6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:57.232303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc3b23-9f4a-4be8-bb8f-3262eeb422d6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:57.264620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:57.265219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:57.265496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:57.265764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:57.266428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:57.267774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883fd2b-fdde-49fb-827a-8a3c2838936c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:57.283608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 581/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Sep 28 07:12:57 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:12:59.436442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47271d-02a6-4c31-92c9-f73401c1d7ce None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:59.437051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47271d-02a6-4c31-92c9-f73401c1d7ce None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:59.437051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47271d-02a6-4c31-92c9-f73401c1d7ce None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:59.438332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 589/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:12:59.443658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:59.444071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:59.444206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:59.461880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:12:59.463056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:12:59.473454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:59.529240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 582/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:12:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:12:59.797703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47271d-02a6-4c31-92c9-f73401c1d7ce None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:12:59.897334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47271d-02a6-4c31-92c9-f73401c1d7ce None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:12:59.913940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:12:59.930893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:12:59.931513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:12:59.931793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:12:59.932319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] PATH_INFO: `/v3/domains/92769e4c648f46ebb41a699d8a5b2c9d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:12:59.943151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=92769e4c648f46ebb41a699d8a5b2c9d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:12:59.944694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0441c1-6d82-4bd4-ad31-2a4a6188c508 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:12:59.946378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 590/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 07:12:59 2022] GET /identity/v3/domains/92769e4c648f46ebb41a699d8a5b2c9d => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:12:59.952269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:00.044807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488aa7-f3c0-40b4-9a0b-0ab159d8d914 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:00.076831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:00.077444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:00.077715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:00.077978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:00.078612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:00.080002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638beda-faa7-47ff-9225-6b38266fb97b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:00.099087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:00.107675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 583/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:12:59 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:13:02.249723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca998ddf-947a-4944-8a2a-a3a2eaab5890 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:02.250287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca998ddf-947a-4944-8a2a-a3a2eaab5890 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:02.250287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca998ddf-947a-4944-8a2a-a3a2eaab5890 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:02.251584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 591/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:13:02.256714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:02.257156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:02.257277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:02.274895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:13:02.276053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:13:02.339717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 584/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:13:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:13:02.612512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca998ddf-947a-4944-8a2a-a3a2eaab5890 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:02.711781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca998ddf-947a-4944-8a2a-a3a2eaab5890 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:02.746303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:02.746922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:02.747202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:02.747474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] PATH_INFO: `/v3/domains/92769e4c648f46ebb41a699d8a5b2c9d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:02.748575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=92769e4c648f46ebb41a699d8a5b2c9d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:02.750030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f63198-4e3c-45f3-b955-3bba3555c39a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:02.751586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 592/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 07:13:02 2022] GET /identity/v3/domains/92769e4c648f46ebb41a699d8a5b2c9d => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:02.757301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:02.852792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a100ba-581f-4c46-8be1-0c86a921be20 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:02.880553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:02.889170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:02.889760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:02.890032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:02.890294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:02.890930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:02.892343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b8695-bfdb-4779-a312-1a907061e164 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:02.913045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 585/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:13:02 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:13:05.086322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1b078-70d4-40ea-afd1-4bc719aecb52 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:05.086839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1b078-70d4-40ea-afd1-4bc719aecb52 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:05.086839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1b078-70d4-40ea-afd1-4bc719aecb52 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:05.088183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 593/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:13:05.094078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:05.094515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:05.094623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:05.112831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:13:05.114020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:13:05.176690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 586/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:13:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:13:05.445589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1b078-70d4-40ea-afd1-4bc719aecb52 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:05.492960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:05.546452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1b078-70d4-40ea-afd1-4bc719aecb52 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:05.578432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:05.579042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:05.579314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:05.579580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:05.589624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:13:05.594285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:13:05.595360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:13:05.596401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 594/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:13:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:13:05.601426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:05.698095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8716-1ca8-4932-afad-6a9627143ad5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:05.711798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:05.733206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:05.733794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:05.734066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:05.734332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:05.734983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:13:05.735279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:05.736624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:05.744268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 587/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:13:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:13:05.750392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:05.859765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf98f-7cae-4abb-a163-3b1182c15094 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:05.867312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:05.893910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:05.894525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:05.894800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:05.895067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] PATH_INFO: `/v3/users/4e02b99951ea4aff86b9405724de84f3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:05.908424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=4e02b99951ea4aff86b9405724de84f3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:05.909887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:05.911461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 595/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:13:05 2022] GET /identity/v3/users/4e02b99951ea4aff86b9405724de84f3 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:05.917356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:06.012705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860121-11eb-46a9-9dbb-da8f5f5d3669 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:06.043982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:06.044559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:06.044830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:06.045121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:06.087398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:06.110088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:06.111501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:06.205013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 588/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:13:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:13:06.210862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:06.309476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad18c47-d6f1-471b-862f-f0f557589fe1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:06.343224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:06.343835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:06.344111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:06.344379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] PATH_INFO: `/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:06.345498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d91bf910b74ea4b1f7c39e2afdfb2a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:06.346904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0e9d1-c61b-4558-88e3-7c53e44d3dc7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:06.348491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 596/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:13:06 2022] GET /identity/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:06.354524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:06.427081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:06.451384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fe25d-746b-4583-9a0b-d41ef5de30dd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:06.483244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:06.483821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:06.484093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:06.484359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:06.485093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=e9d91bf910b74ea4b1f7c39e2afdfb2a, user.id=4e02b99951ea4aff86b9405724de84f3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:13:06.485393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:06.486782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b8f85-08b0-4c25-9deb-39f4d06e53de None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:06.491843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 589/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:13:06 2022] GET /identity/v3/role_assignments?user.id=4e02b99951ea4aff86b9405724de84f3&scope.project.id=e9d91bf910b74ea4b1f7c39e2afdfb2a&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:08.606705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c895c9-a6e6-406c-95e1-fdccfc8091d8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:08.607283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c895c9-a6e6-406c-95e1-fdccfc8091d8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:08.607283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c895c9-a6e6-406c-95e1-fdccfc8091d8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:08.608560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 597/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:13:08.613850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:08.614255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:08.614404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:08.632685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:13:08.633889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:13:08.697682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 590/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:13:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:13:08.969028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c895c9-a6e6-406c-95e1-fdccfc8091d8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:08.999466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:09.066961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c895c9-a6e6-406c-95e1-fdccfc8091d8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:09.099574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:09.100178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:09.100450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:09.100715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:09.110637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:13:09.115307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:13:09.116408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:13:09.117648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 598/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:13:09.122382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:09.188332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:09.218965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3d874-cbef-4217-a2bb-d898df35ffc3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:09.251429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:09.251957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:09.252228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:09.252490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:09.253158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:13:09.253454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:09.254797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:09.262897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 591/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:09.268576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:09.355546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:09.364240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4fb8-9764-41b8-8723-b0f94cd5900b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:09.398172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:09.398782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:09.399055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:09.399319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] PATH_INFO: `/v3/users/4e02b99951ea4aff86b9405724de84f3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:09.400404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] RBAC: Authorizing `identity:get_user(user_id=4e02b99951ea4aff86b9405724de84f3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:09.401953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:09.403495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 599/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:13:09 2022] GET /identity/v3/users/4e02b99951ea4aff86b9405724de84f3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:09.409271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:09.504492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2025788-4fdf-4166-8bff-27a9b38c0676 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:09.531896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:09.537191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:09.537798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:09.538069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:09.538332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] PATH_INFO: `/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:09.539420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] RBAC: Authorizing `identity:get_project(project_id=e9d91bf910b74ea4b1f7c39e2afdfb2a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:09.540820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc533a95-0dbf-4005-a027-fcc83732d2af None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:09.542462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 592/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:13:09 2022] GET /identity/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:09.548335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:09.645221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126faa9-0486-42d2-b8df-9d171a589b95 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:09.679349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:09.679967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:09.680251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:09.680505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] PATH_INFO: `/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a/users/4e02b99951ea4aff86b9405724de84f3/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:09.683535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d91bf910b74ea4b1f7c39e2afdfb2a, user_id=4e02b99951ea4aff86b9405724de84f3, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:09.685055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcae3e8-e566-4668-81d3-29691ccc6b87 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:09.699797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 600/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:13:09 2022] PUT /identity/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a/users/4e02b99951ea4aff86b9405724de84f3/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:13:11.802780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab6d3-8f56-40d5-884a-b30bf3a144fb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:11.803297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab6d3-8f56-40d5-884a-b30bf3a144fb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:11.803297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab6d3-8f56-40d5-884a-b30bf3a144fb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:11.804617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 593/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:13:11.810244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:11.810662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:11.810798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:11.828457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:13:11.829661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:13:11.843649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:11.895887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 601/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:13:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:13:12.163348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceab6d3-8f56-40d5-884a-b30bf3a144fb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:12.264666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceab6d3-8f56-40d5-884a-b30bf3a144fb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:12.298230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:12.298845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:12.299121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:12.299391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:12.309943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:13:12.314726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:13:12.315777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:13:12.316744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 594/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:13:12.322025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:12.417658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab9803-9d2b-4838-b1b8-83b420e6b641 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:12.450777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:12.451370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:12.451649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:12.451922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:12.452575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:13:12.452886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:12.454271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:12.462124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 602/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:13:12.468409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:12.525186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:12.565854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1c39b-4a2b-4bd9-bda1-d86c2c5a84c1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:12.598322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:12.598935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:12.599215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:12.599488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] PATH_INFO: `/v3/users/4e02b99951ea4aff86b9405724de84f3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:12.600592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] RBAC: Authorizing `identity:get_user(user_id=4e02b99951ea4aff86b9405724de84f3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:12.602039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:12.603602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 595/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 07:13:12 2022] GET /identity/v3/users/4e02b99951ea4aff86b9405724de84f3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:12.609485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:12.625221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:12.707779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082cec-e247-43a7-bcb9-b8cfdc8ccb10 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:12.740179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:12.740768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:12.741072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:12.741342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:12.804689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:12.806148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:12.819839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:12.900665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 603/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:13:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:13:12.906981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:13.009621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb166fd4-256b-4a29-99e2-e96749d4cac8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:13.034477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:13.042875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:13.043482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:13.043760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:13.044030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] PATH_INFO: `/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:13.045162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d91bf910b74ea4b1f7c39e2afdfb2a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:13.046579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e74ae-72ec-46c7-95a4-38f89a8fb833 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:13.048180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 596/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 07:13:12 2022] GET /identity/v3/projects/e9d91bf910b74ea4b1f7c39e2afdfb2a => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:13.054320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:13:13.148278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fd8c5-9716-4170-a5f9-be5e443dcce0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:13:13.161138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:13.182928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:13:13.183514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:13:13.183793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:13:13.184062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:13:13.184786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=e9d91bf910b74ea4b1f7c39e2afdfb2a, user.id=4e02b99951ea4aff86b9405724de84f3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:13:13.185126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:13:13.186545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d3ad4-5143-4de7-b1b8-a618ccc68de0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:13:13.192068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 604/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:13:13 2022] GET /identity/v3/role_assignments?user.id=4e02b99951ea4aff86b9405724de84f3&scope.project.id=e9d91bf910b74ea4b1f7c39e2afdfb2a&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:13:23.038305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:13:23.189548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=12 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:09.101216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc86439-1094-4d49-9505-c321ccdb8d3b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:09.101905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc86439-1094-4d49-9505-c321ccdb8d3b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:09.101905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc86439-1094-4d49-9505-c321ccdb8d3b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:09.103765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 597/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:09.110076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:09.110584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:09.110584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:09.130371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:09.131571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:09.215178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 605/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:09.508499 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc86439-1094-4d49-9505-c321ccdb8d3b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:09.658523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc86439-1094-4d49-9505-c321ccdb8d3b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:09.701490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:09.702418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:09.702418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:09.702747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:09.717922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:09.719388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:09.740699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:09.742201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:09.743160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:09.743160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:09.743160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:09.743160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 598/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:09.750623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:09.896718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff6426-333b-4181-bb38-d731062ddd75 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:09.935084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:09.942109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:09.942905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:09.943335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:09.943764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:09.944401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:09.944700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:09.946381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8179a-d4f7-46f6-8d32-6ff41033307f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:09.965216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 606/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:09.971775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:10.037515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:10.102200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8e6aa-bbb7-4b69-a630-2227dd7fec2b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:10.148228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:10.148768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:10.150291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:10.151002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:10.151747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:10.153658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d80e0-06d4-4bf9-9b5c-d0f4a312d5fe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:10.193198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 599/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 07:14:09 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:14:12.813644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b12d8-72e3-495f-944f-561e3e8a719f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:12.814208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b12d8-72e3-495f-944f-561e3e8a719f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:12.814208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b12d8-72e3-495f-944f-561e3e8a719f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:12.815664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 607/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:12.822228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:12.822764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:12.822907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:12.843543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:12.844717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:12.924578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 600/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:13.229533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b12d8-72e3-495f-944f-561e3e8a719f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:13.368045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b12d8-72e3-495f-944f-561e3e8a719f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:13.411477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:13.412745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:13.413385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:13.414013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:13.429297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:13.435338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:13.436459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:13.442332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 608/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:13.443896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:13.490028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:13.560923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55228fa9-5744-45f7-af02-41d27dd4b2af None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:13.606514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:13.607211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:13.607487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:13.607751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:13.608406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:13.608702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:13.610083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:13.621302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 601/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:14:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:13.624101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:13.686489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:13.751558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9daeef-774d-4949-bca1-090b9044d90b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:13.787390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:13.788128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:13.788377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:13.788670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:13.800075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:13.801571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:13.817089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:13.818534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:13.819432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:13.819432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:13.819432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:13.819432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 609/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:14:13.824705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:13.932084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:13.936429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b73314-3be1-478c-87a2-5d220865aadf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:13.977158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:13.978095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:13.978493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:13.978967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:13.980368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:13.981036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:13.982622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:14.002176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 602/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:14.003403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:14.124126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa1ae-48c8-49d9-b786-349599970c47 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:14.126532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:14.167774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:14.168469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:14.168744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:14.169038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:14.178098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:14.179743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:14.193156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:14.194256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:14.195261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 610/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:14.200877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:14.328206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e0b-b6a9-4eaf-966c-b312944dbbff None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:14.367849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:14.368452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:14.368722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:14.369008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:14.369752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:14.370049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:14.371639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:14.395483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 603/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:14.397286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:14.530182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff658b-7fbf-48d4-bf65-127bd87b2328 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:14.570897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:14.571513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:14.571819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:14.572134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:14.606727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:14.653288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:14.654735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:14.772835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 611/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:14:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:14:14.779304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:14.800313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:14.907415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc7845-91ab-4711-b258-086477f6e01f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:14.944652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:14.945310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:14.945583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:14.945908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:14.959184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:14.960759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:14.981684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:14.982896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:14.983933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:14.983933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:14.983933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:14.988414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 604/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:14.989368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:15.078146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:15.127113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2651ca2-2335-428b-b005-721c132c3433 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:15.165954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:15.166480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:15.166758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:15.167160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:15.168011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:15.168376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:15.169771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:15.183408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 612/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:15.189762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:15.282721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:15.331959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d3cd2-0227-4588-adcf-b4d80c62811b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:15.378190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:15.378992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:15.379396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:15.379865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:15.386765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:15.388184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:15.401608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:15.403057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:15.403988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 605/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:15.409630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:15.519332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8b4d-feea-439e-9d60-8d9b9980ea8d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:15.542868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:15.557793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:15.559259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:15.559562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:15.559797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:15.560484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:15.560789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:15.562244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065ae4-d5d1-4d8e-a966-06844624ba56 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:15.574974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 613/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:15.582168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:15.699001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aecf1-b6ee-4448-a1be-96c725a39975 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:15.730456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:15.736585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:15.737475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:15.737970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:15.738308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:15.739483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=b1cbc4b10d4745d08bb05dbd7a36dc84) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:15.739853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:15.741801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c399374-8a98-4832-b454-781aae5016e0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:15.748956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 606/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:14:15 2022] GET /identity/v3/role_assignments?user.id=b1cbc4b10d4745d08bb05dbd7a36dc84&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:17.947802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7058-d534-471a-b470-94dc9e621f9e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:17.948407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7058-d534-471a-b470-94dc9e621f9e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:17.949206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7058-d534-471a-b470-94dc9e621f9e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:17.950464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 614/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:17.955894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:17.956315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:17.956405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:17.979952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:17.981187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:18.067184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 607/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:14:18.350253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7058-d534-471a-b470-94dc9e621f9e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:18.461423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7058-d534-471a-b470-94dc9e621f9e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:18.505031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:18.505689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:18.506002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:18.506316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:18.518111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:18.523109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:18.524597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:18.525833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 615/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:14:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:18.531097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:18.642046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e062d2d-4fc3-41f8-8507-63ad6e65400b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:18.666901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:18.678734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:18.679358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:18.679672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:18.679940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:18.680737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:18.680862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:18.682323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:18.692097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 608/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:14:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:18.698006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:18.748388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:18.810662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6e38-2009-452e-8468-fe16b8fe64e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:18.847818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:18.848449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:18.848726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:18.849031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:18.860350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:18.862142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:18.880327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:18.881654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:18.882789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:18.882789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:18.882789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:18.882789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 616/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:18.888095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:18.991474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b61a-3cab-464e-8497-a2c5722f2b1c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.028983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.029598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.029901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.030156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.030883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:19.031189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.032704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.046554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 609/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:19.052496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:19.158692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:19.167570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2371a26-f169-48fc-870f-fe8f4086f319 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.205330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.205996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.206403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.206578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.214536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.216157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:19.228812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:19.230460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:19.231520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 617/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:19.237384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:19.294834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:19.360441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c66a7-1214-4988-8db5-c5b65f6a6d34 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.399514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07346-8538-4666-bac3-b92445623976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.400112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07346-8538-4666-bac3-b92445623976 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.400524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07346-8538-4666-bac3-b92445623976 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.400794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07346-8538-4666-bac3-b92445623976 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.402192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07346-8538-4666-bac3-b92445623976 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:19.402595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07346-8538-4666-bac3-b92445623976 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.404250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07346-8538-4666-bac3-b92445623976 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.423385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 610/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:19.429463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:19.536612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902e51b-67b6-4f2b-91f3-59f556f060ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.572561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.573158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.573464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.573832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.584374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.585859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:19.600540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:19.601837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:19.603335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:19.603335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:19.603335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:19.607544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 618/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:19.608443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07346-8538-4666-bac3-b92445623976 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:19.718954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07346-8538-4666-bac3-b92445623976 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.721306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:19.761569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.762181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.762457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.762786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.763430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:19.763804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.765206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.778486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 611/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:14:19.784246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:19.794383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:19.894484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f4d0-0edb-4b03-82c5-05abc7ab9443 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:19.930597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:19.931225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:19.931504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:19.931775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:19.943097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:19.944534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:19.958373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:19.959511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:19.960551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 619/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:19.966523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:20.074967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1945-7846-4b84-af3d-90d6908a9f9e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:20.112964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:20.113728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:20.114002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:20.114278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:20.114967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:20.115267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:20.116694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16d506-d843-4c79-b99c-2afff9f43c18 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:20.131079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 612/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:20.139791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:20.217969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:20.259298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535a138-4836-445f-80e7-6140dfeff034 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:20.301151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:20.301867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:20.302175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:20.302481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/b1cbc4b10d4745d08bb05dbd7a36dc84/roles/843209adb8de416aa72cacc0023709ed` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:20.321241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=b1cbc4b10d4745d08bb05dbd7a36dc84, role_id=843209adb8de416aa72cacc0023709ed)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:20.322840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090ffd-4e06-4f28-8513-761459bc079f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:20.345811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 620/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:14:20 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/b1cbc4b10d4745d08bb05dbd7a36dc84/roles/843209adb8de416aa72cacc0023709ed => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:14:22.782953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d281-eb3c-431f-84a6-32cbd7c4c467 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:22.783522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d281-eb3c-431f-84a6-32cbd7c4c467 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:22.783522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d281-eb3c-431f-84a6-32cbd7c4c467 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:22.784881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 613/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:22.790389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434edc34-92dc-48c8-986b-6210dad854da None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:22.790867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434edc34-92dc-48c8-986b-6210dad854da None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:22.791127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434edc34-92dc-48c8-986b-6210dad854da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:22.813191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-434edc34-92dc-48c8-986b-6210dad854da None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:22.814759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-434edc34-92dc-48c8-986b-6210dad854da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:22.885861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 621/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:23.171501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d281-eb3c-431f-84a6-32cbd7c4c467 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:23.200663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:23.295809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d281-eb3c-431f-84a6-32cbd7c4c467 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:23.347709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:23.348731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:23.349175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:23.349677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:23.364426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:23.371701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:23.372746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:23.374411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 614/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 07:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:23.379954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434edc34-92dc-48c8-986b-6210dad854da None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:23.434028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:23.494405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434edc34-92dc-48c8-986b-6210dad854da None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:23.535225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:23.535844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:23.536124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:23.536458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:23.537249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:23.537498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:23.538972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:23.552481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 622/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 07:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:14:23.560020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:23.655708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:23.673883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4af1-9553-4c09-9d88-589b7556f7c8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:23.716565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:23.717272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:23.717546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:23.717814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:23.728219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:23.729870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:23.746729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:23.747875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:23.748774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:23.748774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:23.748774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:23.748774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 615/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:23.753834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:23.866867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9875947-21e1-4750-ba45-b0158cc86995 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:23.879746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:23.908178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:23.908787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:23.909138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:23.909407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:23.910041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:23.910363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:23.911815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:23.924007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 623/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:23.930220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:24.044009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fd9c7-ac99-43cb-8e01-590016ed7cb1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:24.083198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:24.084168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:24.084646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:24.085040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:24.088476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:24.095239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:24.097117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:24.108585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:24.109757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:24.111334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 616/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 07:14:24.116702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:24.235122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbc025-ea44-441b-a975-9b589b439e52 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:24.275311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:24.275903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:24.276176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:24.276478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:24.277350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:24.277648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:24.279093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:24.297263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 624/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:24.303629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:24.422278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73240ac3-f1d0-4e66-91da-e61af2cab775 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:24.458355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:24.459096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:24.459395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:24.459669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:24.531709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:24.539066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:24.540652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:24.650578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 617/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:14:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:14:24.657418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:24.690102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:24.771063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4f44-3f4b-43a8-9096-64cb8444ccb7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:24.811022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:24.811719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:24.811995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:24.812318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:24.823120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:24.824923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:24.842481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:24.843516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:24.844496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:24.844496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:24.844496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:24.844496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 625/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:24.849814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:24.953023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:24.959903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518c61-4198-4827-b95a-3c5949ae9706 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:24.998651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:24.999252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:24.999528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:24.999792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:25.000418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:25.000719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:25.002098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:25.013554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 618/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:25.019527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:25.111157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:25.133911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966ccd-5a30-4b6c-a96a-d309f558ecdb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:25.175042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:25.175672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:25.175970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:25.176211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:25.184276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:25.185825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:25.197029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:25.198389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:25.200017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 626/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:25.207216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:25.321366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0946-1f9e-4442-a9d3-1be0059f7335 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:25.360356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:25.361006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:25.361323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:25.361627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:25.362375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:25.362719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:25.364132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1ee1a-2b0b-4bf6-ac00-7c5522bf0902 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:25.377589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 619/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:25.384371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:25.495845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123214d7-c018-44b4-899e-292cfb3a46f8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:25.532422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:25.533059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:25.533335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:25.533601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:25.534328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] RBAC: Adding query filter params (role.id=843209adb8de416aa72cacc0023709ed, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=b1cbc4b10d4745d08bb05dbd7a36dc84) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:25.534634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:25.536059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e5690-5f5e-4835-95fa-661160283cbf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:25.542703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 627/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:14:25 2022] GET /identity/v3/role_assignments?user.id=b1cbc4b10d4745d08bb05dbd7a36dc84&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=843209adb8de416aa72cacc0023709ed => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:28.003206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0cf4-ccf9-4f95-b898-8bee9ef1aed0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:28.005142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0cf4-ccf9-4f95-b898-8bee9ef1aed0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:28.005142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0cf4-ccf9-4f95-b898-8bee9ef1aed0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:28.005142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 620/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:28.011638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:28.012101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:28.012739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:28.020692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:28.035808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:28.036919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:28.114760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 628/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:28.411598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0cf4-ccf9-4f95-b898-8bee9ef1aed0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:28.430702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:28.519107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0cf4-ccf9-4f95-b898-8bee9ef1aed0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:28.554389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:28.555022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:28.555294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:28.555559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:28.567168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:28.572010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:28.573166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:28.575042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 621/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:28.580148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:28.683690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b3d11-8e9c-4416-8ad5-d64f966de3d0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:28.717246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:28.723324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:28.723934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:28.724210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:28.724480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:28.725175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:28.725476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:28.726833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:28.735591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 629/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:28.741768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:28.825169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:28.847991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09f00e-7643-4364-b193-c9594c9b97de None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:28.883836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:28.884462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:28.884735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:28.885040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:28.895956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:28.897496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:28.914140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:28.915110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:28.916332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:28.916332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:28.916332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:28.916332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 622/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:28.921506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.025911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1181593-db68-4c4b-8be9-9d2013861472 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.058306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.059136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.059409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.059673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.060294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:29.060589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:29.061963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:29.072976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 630/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:29.079170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.175888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757f266-be85-41d7-9554-dadf15af544a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.202613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:29.210684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.211296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.211566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.211829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.219256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:29.220672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:29.231344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:29.232403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:29.233425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 623/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:29.238220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.291932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:29.346994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25fa26-9bc5-4bfd-a47d-b9cbb086fb5c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.379899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.380487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.380760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.381064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.381778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:29.382079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:29.383486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:29.400303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 631/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:29.406288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.501692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecc069-1b36-4230-bbcb-36186786d01d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.533957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.534543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.534820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.535090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.592167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:29.604229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:29.606205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:29.700388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 624/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:14:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:14:29.706136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.801074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefb6c7-0d23-43f7-b51e-ce60c9b94d85 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.814388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:29.834685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.835293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.835568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.835835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.845514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:29.846887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:29.861367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:29.862510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:29.863819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:29.863819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:29.863819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:29.863819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 632/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:29.868036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:29.952455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:29.964254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa994953-d988-40c3-82cc-461ca35faec1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:29.997908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:29.998506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:29.998781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:29.999047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:29.999677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:29.999972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:30.001343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:30.012970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 625/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:30.018502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:30.112853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963852f-abe2-4815-8759-7601c6423158 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:30.145475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:30.146089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:30.146364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:30.146634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:30.153219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:30.154647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:30.165558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:30.166582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:30.167842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 633/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:30.172457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:30.269041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60e907-1b12-4053-9a30-336091c61c4f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:30.302569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:30.303198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:30.303539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:30.303859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:30.304607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:30.305171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:30.306748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b2fe7-593b-4623-9855-2e04107d522d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:30.322136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 626/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:30.329087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:30.355297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:30.426661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811035f-dee5-4320-acf7-c4d4fc30618b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:30.458621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:30.459202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:30.459474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:30.459739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:30.460441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=b1cbc4b10d4745d08bb05dbd7a36dc84) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:30.460740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:30.462183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af01200-18c1-4cfd-98af-c1a0a38f3ca7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:30.467273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 634/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:14:30 2022] GET /identity/v3/role_assignments?user.id=b1cbc4b10d4745d08bb05dbd7a36dc84&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:32.597915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5ace-fcee-4e7a-9ce1-5d4cead9213a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:32.598510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5ace-fcee-4e7a-9ce1-5d4cead9213a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:32.598510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5ace-fcee-4e7a-9ce1-5d4cead9213a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:32.599776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 627/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:32.605716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:32.606218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:32.606341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:32.623787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:32.624970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:32.688547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 635/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:32.957497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5ace-fcee-4e7a-9ce1-5d4cead9213a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:32.968162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.059707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5ace-fcee-4e7a-9ce1-5d4cead9213a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.092929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.093586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.093865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.094134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.103825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:33.108958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:33.110128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:33.110955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 628/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:33.116006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:33.134097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.213123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143930a-7835-48b0-a2d6-15ff5e479a08 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.251192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.251783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.252060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.252324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.252996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:33.253297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:33.254652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:33.262888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 636/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:33.268763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:33.349243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.387779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd6d5-e373-4cb7-9d17-2d334a9160be None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.420231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.420837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.421140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.421405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.430951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:33.432374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:33.446710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:33.447818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:33.449791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:33.449791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:33.449791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:33.453098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 629/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:33.454139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:33.531709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.550557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bf28-8677-4bf2-b49c-38ee1b19a608 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.582943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.583526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.583799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.584065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.584686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:33.585011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:33.586350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:33.598364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 637/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:33.603992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:33.702185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681fb48-65a2-4db7-82d2-64825d058248 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.721754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.736093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.736710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.737019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.737294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.745877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:33.747520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:33.758405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:33.759552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:33.761404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 630/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:33.766206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:33.865266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a9b9-213d-4cd7-81ae-1516a61d951b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:33.901046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:33.901917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:33.902223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:33.902494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:33.903212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:33.903517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:33.904927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:33.913248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:33.922198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 638/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:33.927898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:34.023370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f575-66e1-4fe8-8ecc-5b692e856da0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:34.054775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:34.055378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:34.055647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:34.055906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:34.066916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:34.068326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:34.081852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:34.082964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:34.084094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:34.084094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:34.084094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:34.084094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 631/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:34.088418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:34.184648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4969b-6919-4814-a501-272ae866ce3f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:34.217526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:34.218113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:34.218389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:34.218658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:34.219288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:34.219586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:34.220927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:34.232316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 639/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:34.237994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:34.268455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:34.337497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65b07-2a76-499c-8a6d-389a3fe2d377 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:34.369992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:34.371211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:34.371481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:34.371747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:34.378040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:34.379445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:34.390373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:34.391523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:34.392601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 632/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:34.397304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:34.454601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:34.495599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed66e4-39b1-430c-8a03-d8fb394cec78 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:34.527802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:34.528384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:34.528661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:34.528929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:34.529636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:34.529940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:34.531349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbab1f-7e11-4128-8079-64440a09f70c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:34.543592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 640/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:34.549435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:34.636478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:34.645473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f4c4a-0842-4563-aba0-dbabf5fe65d7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:34.678024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:34.678653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:34.678927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:34.679194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] PATH_INFO: `/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/b1cbc4b10d4745d08bb05dbd7a36dc84/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:34.681063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] RBAC: Authorizing `identity:create_grant(project_id=a8c197c9445d453dbdf730b9d157cf14, user_id=b1cbc4b10d4745d08bb05dbd7a36dc84, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:34.682523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a001cc-a0e0-4833-8ef6-7b5ec6f73fec None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:34.697231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 633/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 07:14:34 2022] PUT /identity/v3/projects/a8c197c9445d453dbdf730b9d157cf14/users/b1cbc4b10d4745d08bb05dbd7a36dc84/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:14:36.791044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb56e3-5fac-4389-af5a-aa2959dd0464 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:36.791564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb56e3-5fac-4389-af5a-aa2959dd0464 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:36.791564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb56e3-5fac-4389-af5a-aa2959dd0464 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:36.793594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 641/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:36.798630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:36.799040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:36.799170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:36.818335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:36.819504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:36.882721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 634/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:37.147801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb56e3-5fac-4389-af5a-aa2959dd0464 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.246323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb56e3-5fac-4389-af5a-aa2959dd0464 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:37.263178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:37.280085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:37.280694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:37.280986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:37.281252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:37.290818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 07:14:37.295911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:37.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:37.298116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 642/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:37.303578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.394620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:37.413466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f6b16-e1f1-42d0-a839-12a24f274429 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:37.446797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:37.447390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:37.447665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:37.447933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:37.449274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:37.449575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:37.450924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:37.458551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 635/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:37.464384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.560209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdf3a2-85dc-47b7-afdb-c5fe175a123e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:37.593089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:37.593698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:37.593974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:37.594244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:37.603936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:37.605728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:37.619800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:37.621515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:37.622620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:37.622620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:37.622620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:37.622620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 643/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:37.627194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.722682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09357-4c25-451d-a9f0-0cffb8e75f1f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:37.754514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:37.755101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:37.755370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:37.755634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:37.756284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:37.756576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:37.757946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:37.769691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 636/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:37.775358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.805133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:37.873193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba2a64-ccc0-46f0-a2cb-6771b6397a69 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:37.904950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:37.905577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:37.905853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:37.906123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:37.913858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:37.915277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 07:14:37.926561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:37.927643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:37.928762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 644/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:37.934003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:37.938030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.028680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ae5f-6db3-4fd9-8675-f36c0159f55e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.062596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.063184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.063460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.063729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.064441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:38.064742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.066193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:38.082605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 637/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 07:14:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:38.088287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:38.176602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.186714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f004-d7d6-405a-b1fb-ef6dea62bc9e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.219279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.219869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.220141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.220403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.283842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.285218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:38.369948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.381378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 645/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 07:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 07:14:38.387257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:38.446591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.484552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb81c9a-95ad-46d0-9c62-a41b2b1a2665 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.516745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.517381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.517659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.517932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.528174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.529632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 07:14:38.544278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:38.545388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 07:14:38.546491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 07:14:38.546491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 07:14:38.546491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:38.546491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 638/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:38.551021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:38.645993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37322135-758e-4156-aabc-0e158d47e267 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.678852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.679437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.679711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.680075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.680700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:38.681020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.682366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:38.693440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 646/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 07:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:38.699732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:38.795738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb4a41-b57b-4ccf-8f30-e2c963ac1e57 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.808668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.828981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.829579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.829851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.830117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.836407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.837847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 07:14:38.850216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:38.851232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Sep 28 07:14:38.852168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 639/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 07:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:38.857243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:38.906054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:38.958407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9b3b0-f47b-4a8b-bc15-a04424626642 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:38.992711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:38.993329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:38.993604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:38.993873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:38.994550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:38.994851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:38.997075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98aa16-91fa-444e-b40f-a7c5938d1650 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:39.008497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 647/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 07:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:39.015203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:39.109934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9ceba-3ed1-466f-8a41-da0aad33dcd8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:39.142002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:39.142575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:39.142847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:39.143113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:39.143814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] RBAC: Adding query filter params (role.id=30776d1972a74ad8b38be970aa4d6478, scope.project.id=a8c197c9445d453dbdf730b9d157cf14, user.id=b1cbc4b10d4745d08bb05dbd7a36dc84) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:39.144114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:39.145548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07244b6-5442-4c93-879a-05713f645993 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:39.150951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 640/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 07:14:39 2022] GET /identity/v3/role_assignments?user.id=b1cbc4b10d4745d08bb05dbd7a36dc84&scope.project.id=a8c197c9445d453dbdf730b9d157cf14&role.id=30776d1972a74ad8b38be970aa4d6478 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:41.284414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c8943-68a8-45e8-ac6e-e11c9f3ff094 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:41.284972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c8943-68a8-45e8-ac6e-e11c9f3ff094 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:41.284972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c8943-68a8-45e8-ac6e-e11c9f3ff094 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:41.286294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 648/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:41.291607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:41.292051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:41.292183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:41.313071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:41.314301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:41.348235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:41.389274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 641/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:14:41.658275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c8943-68a8-45e8-ac6e-e11c9f3ff094 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:41.757190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c8943-68a8-45e8-ac6e-e11c9f3ff094 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:41.768337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:41.792045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:41.792693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:41.792988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:41.793262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:41.793837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:41.795181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:41.803657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:41.804892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:41.805988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 649/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:14:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:41.811301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:41.905774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee6401-40ec-4ef5-b1cd-1804f4454de4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:41.938272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:41.938797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:41.939070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:41.939338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:41.939958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:41.940257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:41.941625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9efefb-0f6f-4416-a462-0b6d986cabb5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:41.946216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 642/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 07:14:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:41.952368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:42.048024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ce1dc-0819-40e8-a1a6-c3b37a61c5a0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:42.079611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:42.080183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:42.080457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:42.080723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:42.081363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:42.081659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:42.082993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49b1de-dae4-4ad3-9c06-d8866795a761 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:42.087195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 650/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 07:14:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:44.195027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654780e-ccc1-4020-9ffa-6d4a26f15a08 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:44.195590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654780e-ccc1-4020-9ffa-6d4a26f15a08 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:44.195590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654780e-ccc1-4020-9ffa-6d4a26f15a08 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:44.196888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 643/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:44.202667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c84b-2c80-47ac-a0de-aaddc87642af None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:44.203041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c84b-2c80-47ac-a0de-aaddc87642af None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:44.203216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c84b-2c80-47ac-a0de-aaddc87642af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:44.220859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ee23c84b-2c80-47ac-a0de-aaddc87642af None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:44.222045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ee23c84b-2c80-47ac-a0de-aaddc87642af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:44.250620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:44.286733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 651/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:44.552493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654780e-ccc1-4020-9ffa-6d4a26f15a08 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:44.583387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:44.651550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654780e-ccc1-4020-9ffa-6d4a26f15a08 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:44.685096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:44.685681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:44.685956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:44.686226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:44.686886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:44.688262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e99b-f544-4470-8695-75ef8ea45e31 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:44.703126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 644/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 07:14:44 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:14:46.955783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfbc4-26c5-426f-a7b8-82a2c3842d11 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:46.956291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfbc4-26c5-426f-a7b8-82a2c3842d11 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:46.956291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfbc4-26c5-426f-a7b8-82a2c3842d11 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:46.957591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 652/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:46.962940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:46.963342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:46.963563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:46.982161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:46.983339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:47.052710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 645/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 07:14:47.322273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfbc4-26c5-426f-a7b8-82a2c3842d11 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:47.424147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfbc4-26c5-426f-a7b8-82a2c3842d11 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:47.457963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:47.458577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:47.458854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:47.459123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:47.459694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:47.461140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:47.469911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:47.471064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:47.471941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 653/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:14:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:47.477087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:47.494513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:47.574449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352841-f8bb-43ab-8c3d-11718092fec9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:47.607434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:47.608017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:47.608292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:47.608559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:47.609205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:47.609505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:47.610853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:47.615734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 646/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 07:14:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:47.622480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:47.653041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:47.720619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096049b-25d2-437a-a1ac-11de34cf96ed None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:47.753959 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:47.754557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:47.754835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:47.755104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:47.755798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] RBAC: Adding query filter params (interface=public, service_id=ede055ff4c19490aa52f4e4150ddf4fc, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:47.756104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:47.757499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b213-a571-4b8a-a1da-0b374125456f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:47.762345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 654/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 07:14:47 2022] GET /identity/v3/endpoints?service_id=ede055ff4c19490aa52f4e4150ddf4fc&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:47.768080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:47.845609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:47.863114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1d665-67ad-4b5b-a4e9-2e1337f215ee None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:47.895086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:47.895668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:47.895939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:47.896204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:47.896790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:47.898154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d0885-7cdb-4177-951b-c38088ec3a1f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:47.905136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 647/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 07:14:47 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:14:50.038302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd5d44-6b4e-47ad-8f90-7f4ccc70e548 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:50.038899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd5d44-6b4e-47ad-8f90-7f4ccc70e548 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:50.038899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd5d44-6b4e-47ad-8f90-7f4ccc70e548 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:50.040169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 655/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:14:50.045990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:50.046452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:50.046553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:50.064967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:14:50.066148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:14:50.129836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 648/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:14:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 07:14:50.397404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd5d44-6b4e-47ad-8f90-7f4ccc70e548 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:50.489667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:50.496422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd5d44-6b4e-47ad-8f90-7f4ccc70e548 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:50.528965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:50.529575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:50.529848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:50.530112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:50.530677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:50.532026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 07:14:50.540435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 07:14:50.541726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Sep 28 07:14:50.543128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 656/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 07:14:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 07:14:50.548027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:50.621529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:14:50.644504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d4268-de61-49fc-9188-4a47dd818114 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:50.677262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:50.677788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:50.678060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:50.678325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:50.678937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:14:50.679233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:50.680575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e2085-bc5c-46d7-a34f-46ff486d22a9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:50.685844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 649/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 07:14:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:14:50.691720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:14:50.791203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bce23-b6c9-4556-b1b0-8d7b1a583af6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:14:50.824173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:14:50.824754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:14:50.825063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:14:50.825337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:14:50.825997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:14:50.827353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ac3f7-5ca4-48b2-bed8-e728772f20c6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:14:50.846665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 657/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:14:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:15:00.683219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=30 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:15:00.842915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:15:54.971298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496d744-338c-4d17-9030-e4140d3bbd0b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:54.972258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496d744-338c-4d17-9030-e4140d3bbd0b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:54.972783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496d744-338c-4d17-9030-e4140d3bbd0b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:54.982546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79476af-49e4-41f9-a1f3-432675b35e3c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:54.983580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79476af-49e4-41f9-a1f3-432675b35e3c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:54.984160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79476af-49e4-41f9-a1f3-432675b35e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:54.987762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 650/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:15:54 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:15:55.094062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b79476af-49e4-41f9-a1f3-432675b35e3c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:15:55.095456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b79476af-49e4-41f9-a1f3-432675b35e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:55.213070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 658/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:15:55.509034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:55.509332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:55.509980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:55.511555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 651/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 07:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:15:55.518400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e0c0-6f5f-401b-9dd8-cd4fcfa49e28 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:55.518697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e0c0-6f5f-401b-9dd8-cd4fcfa49e28 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:55.519092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e0c0-6f5f-401b-9dd8-cd4fcfa49e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:55.607405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9d61e0c0-6f5f-401b-9dd8-cd4fcfa49e28 None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:15:55.608702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9d61e0c0-6f5f-401b-9dd8-cd4fcfa49e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:55.632991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:15:55.698791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 659/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 07:15:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:15:55.707809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:15:55.709585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:55.830642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b112-f258-45c0-b674-09c80481e4eb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:15:55.912605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:15:55.913183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:55.913457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:55.913720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:55.915339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:55.976174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:15:56.036567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:15:56.038058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7cd2-0309-4a20-957b-bf8e20b03196 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:15:56.058916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 652/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:15:58.492976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:58.493504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:58.493504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:58.495593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 660/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:15:58.500997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c380cd9-36fd-40b1-9a1f-ea9e0763b812 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:58.501412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c380cd9-36fd-40b1-9a1f-ea9e0763b812 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:58.501543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c380cd9-36fd-40b1-9a1f-ea9e0763b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:58.581466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6c380cd9-36fd-40b1-9a1f-ea9e0763b812 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:15:58.582593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c380cd9-36fd-40b1-9a1f-ea9e0763b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:58.686684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 653/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:15:58.966932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:15:58.968451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:59.073793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fde3a3-8d4f-48a9-b70d-a7338ae8dc92 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:15:59.143111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:15:59.145743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:15:59.146694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:15:59.147121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:15:59.147585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:15:59.149217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:15:59.258407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:15:59.259828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a06270-c066-4a89-a04d-669c2e683f12 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:15:59.277746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 661/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:01.744516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:01.745760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:01.745760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:01.747051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 654/1315] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:01.752311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d866a-ae95-4248-84b5-df6127213213 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:01.752742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d866a-ae95-4248-84b5-df6127213213 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:01.752859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d866a-ae95-4248-84b5-df6127213213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:01.818832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e76d866a-ae95-4248-84b5-df6127213213 None None] MFA Rules not processed for user `86e8ac28358642158fd74d784c25c90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:01.820048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e76d866a-ae95-4248-84b5-df6127213213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:01.871845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:01.915659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 662/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:02.199890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:02.201431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:02.211121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:02.306810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2ab63-a028-4da9-94d9-9c4df91968c1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:02.379158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:02.379747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:02.380015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:02.380273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:02.381866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:02.441996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:02.488872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:02.490262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b45871-a17c-4135-8799-b5a8b2d118af service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:02.508461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 655/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:05.177324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385131-a6ac-41ea-9934-88543612d4c0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:05.177903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385131-a6ac-41ea-9934-88543612d4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:05.177903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385131-a6ac-41ea-9934-88543612d4c0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:05.179239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 663/1318] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:05.185119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1310f-8780-4cb9-b4fb-6f509c1387d3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:05.185551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1310f-8780-4cb9-b4fb-6f509c1387d3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:05.185664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1310f-8780-4cb9-b4fb-6f509c1387d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:05.251167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-53c1310f-8780-4cb9-b4fb-6f509c1387d3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:05.252347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-53c1310f-8780-4cb9-b4fb-6f509c1387d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:05.348345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 656/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:05.652593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:05.652869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:05.653165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:05.654551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 664/1320] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:05.660113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b3ca1-8964-4c89-a8ff-975072556e9d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:05.660435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b3ca1-8964-4c89-a8ff-975072556e9d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:05.660710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b3ca1-8964-4c89-a8ff-975072556e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:05.683309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:05.737440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-659b3ca1-8964-4c89-a8ff-975072556e9d None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:05.738629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-659b3ca1-8964-4c89-a8ff-975072556e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:05.813076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 657/1321] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:05.822037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:05.823549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:05.925283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf2bfd-4080-45c5-85b5-2fb6bd6c884b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:05.996604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:05.997232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:05.997509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:05.997776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:05.999351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:06.006003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:06.106936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:06.108360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba1db7-1e58-4847-b10c-55aec0dd3dd1 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:06.125712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 665/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:09.441690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:09.442232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:09.442232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:09.443722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 658/1323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:09.450516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7230f2-5273-4453-849e-7af28e29daef None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:09.451053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7230f2-5273-4453-849e-7af28e29daef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:09.451053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7230f2-5273-4453-849e-7af28e29daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:09.529330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa7230f2-5273-4453-849e-7af28e29daef None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:09.530610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa7230f2-5273-4453-849e-7af28e29daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:09.596232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:09.639943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 666/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:09.930891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:09.932676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:10.047705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447d429-e075-44f1-86c6-382f48c9a94d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:10.066588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:10.133622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:10.134341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:10.135514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:10.135514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:10.137302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:10.245439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:10.246998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365e86-edf5-4928-9cf7-54be626678c7 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:10.266480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 659/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:12.499991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:12.500547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:12.500547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:12.501967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 667/1326] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:12.507224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9263b8-a02e-4cc7-b344-fb3afe32c2a6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:12.507615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9263b8-a02e-4cc7-b344-fb3afe32c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:12.507785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9263b8-a02e-4cc7-b344-fb3afe32c2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:12.556227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:12.575043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cf9263b8-a02e-4cc7-b344-fb3afe32c2a6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:12.576232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf9263b8-a02e-4cc7-b344-fb3afe32c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:12.684521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 660/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:16:12.966742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:12.968289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:13.072749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a6add-f69f-4a3b-9eed-16a1ca5a44e8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:13.148096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:13.148976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:13.149264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:13.149536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:13.151682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:13.169091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:13.260907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:13.262166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ecee-d22a-40ca-a971-f6f0ed76f85f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:13.280467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 668/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:15.485192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:15.485706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:15.485706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:15.486992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 661/1329] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:15.492327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9f6f5-51a5-40a3-8b9b-49e33c739571 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:15.492760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9f6f5-51a5-40a3-8b9b-49e33c739571 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:15.492880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9f6f5-51a5-40a3-8b9b-49e33c739571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:15.558952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8df9f6f5-51a5-40a3-8b9b-49e33c739571 None None] MFA Rules not processed for user `86e8ac28358642158fd74d784c25c90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:15.560077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8df9f6f5-51a5-40a3-8b9b-49e33c739571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:15.626306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:15.658713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 669/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:15.941321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:15.942912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:16.056759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6427e-646e-454e-87e3-3abcc915fce8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:16.078757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:16.132745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:16.133398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:16.133667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:16.133928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:16.136148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:16.243372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:16.245394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f427bf-9bfe-464c-96e3-8af9879f9556 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:16.263748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 662/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:18.659209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:18.659735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:18.659735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:18.661034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 670/1332] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:18.666760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dccffe-05ff-48fc-8595-1883d8cf4e94 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:18.667289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dccffe-05ff-48fc-8595-1883d8cf4e94 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:18.667409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dccffe-05ff-48fc-8595-1883d8cf4e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:18.730416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:18.733123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e8dccffe-05ff-48fc-8595-1883d8cf4e94 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:18.734819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e8dccffe-05ff-48fc-8595-1883d8cf4e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:18.827776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 663/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:16:19.116295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:19.117850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:19.218536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8251da-e71b-4463-90d5-a4eb055613c4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:19.291315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:19.292505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:19.292778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:19.293072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:19.294650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:19.321428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:19.400421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:19.402318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1059a6a-bb08-4592-86f3-fc2598826978 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:19.420525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 671/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:21.608355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:21.608935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:21.608935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:21.610287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 664/1335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:21.615757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969fd7c-85f4-42c7-86b0-172b1354b80a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:21.616198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969fd7c-85f4-42c7-86b0-172b1354b80a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:21.616452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969fd7c-85f4-42c7-86b0-172b1354b80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:21.681332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b969fd7c-85f4-42c7-86b0-172b1354b80a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:21.683036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b969fd7c-85f4-42c7-86b0-172b1354b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:21.757489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:21.779675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 672/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:22.064126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:22.065696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:22.166882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61563d-9d8e-44db-bb3e-ad93ecd2de2e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:22.196181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:22.239766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:22.240359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:22.240631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:22.240896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:22.242486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:22.347989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:22.349371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d17a-f51f-4bdc-8f80-dde66620f2ce service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:22.366714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 665/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:24.844006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:24.844567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:24.844567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:24.845918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 673/1338] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:24.851079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a165ba-dd46-4d77-80e2-f058460a952d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:24.851484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a165ba-dd46-4d77-80e2-f058460a952d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:24.851651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a165ba-dd46-4d77-80e2-f058460a952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:24.916158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-01a165ba-dd46-4d77-80e2-f058460a952d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:24.917926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01a165ba-dd46-4d77-80e2-f058460a952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:24.926020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:25.015610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 666/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:25.303204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:25.304909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:25.407699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70406a6-6d3f-4138-b79a-699c042aaf7d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:25.480798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:25.481425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:25.481696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:25.481961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:25.483539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:25.519477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:25.594416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:25.595766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d2f55-0acc-4b8c-953b-ad9abe81fc2f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:25.613967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 674/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:29.864197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:29.865277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:29.865277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:29.869377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 667/1341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:16:29.875013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e1c3-1101-453c-a879-7ec6697fc43b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:29.875538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e1c3-1101-453c-a879-7ec6697fc43b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:29.876036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e1c3-1101-453c-a879-7ec6697fc43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:29.950339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a1a6e1c3-1101-453c-a879-7ec6697fc43b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:29.951563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1a6e1c3-1101-453c-a879-7ec6697fc43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:30.035876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:30.047696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 675/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:30.333702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:30.335212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:30.435819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f66f3-5a91-411d-b0c9-dbeb64d252e6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:30.474289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:30.513220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:30.513824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:30.514097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:30.514361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:30.515956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:30.627736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:30.629146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13626a35-facd-40dd-8dad-4903c0cb7f3e service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:30.650317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 668/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:33.400505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:33.401642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:33.401642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:33.403145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 676/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:33.410263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0d4e7-8bc7-4719-b632-5cbed274db7b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:33.410707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0d4e7-8bc7-4719-b632-5cbed274db7b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:33.410945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0d4e7-8bc7-4719-b632-5cbed274db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:33.477105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0bf0d4e7-8bc7-4719-b632-5cbed274db7b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:33.478338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0bf0d4e7-8bc7-4719-b632-5cbed274db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:33.498238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:33.582656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 669/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:33.868494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:33.870495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:33.974299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2df00f-7758-4a3e-bde5-f9a3c66fdb56 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:34.045789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:34.046385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:34.046657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:34.046923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:34.048496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:34.093506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:34.158859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:34.160330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f47ae-3d64-4616-b3df-120ab72a6b58 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:34.178257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 677/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:38.630534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:38.631024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:38.631119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:38.632410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 670/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:38.637784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021e43c-2ac1-4adf-892a-a58d02383698 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:38.638209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021e43c-2ac1-4adf-892a-a58d02383698 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:38.638324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021e43c-2ac1-4adf-892a-a58d02383698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:38.712822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c021e43c-2ac1-4adf-892a-a58d02383698 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:38.714984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c021e43c-2ac1-4adf-892a-a58d02383698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:38.810257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:38.815040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 678/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:39.107423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:39.109053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:39.214117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7265-f199-4f87-8cad-b83aedf0ee08 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:39.261061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:39.287257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:39.287855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:39.288131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:39.288403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:39.290021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:39.412964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:39.414310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189942c-b8cb-431f-a35b-2b1ae07c57f9 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:39.432433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 671/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:44.353159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd5dd-87be-4408-9017-03573d4d8230 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:44.353640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd5dd-87be-4408-9017-03573d4d8230 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:44.353747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd5dd-87be-4408-9017-03573d4d8230 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:44.356108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 679/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:44.361405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978123ca-c86f-4994-aba2-dd6ae27dc69c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:44.361841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978123ca-c86f-4994-aba2-dd6ae27dc69c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:44.361978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978123ca-c86f-4994-aba2-dd6ae27dc69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:44.463230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-978123ca-c86f-4994-aba2-dd6ae27dc69c None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:44.464584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-978123ca-c86f-4994-aba2-dd6ae27dc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:44.517303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:44.636749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 672/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:16:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:48.675038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18571213-4809-4e3a-b325-ddfeaad58a37 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:48.675872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18571213-4809-4e3a-b325-ddfeaad58a37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:48.676139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18571213-4809-4e3a-b325-ddfeaad58a37 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:48.677653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 680/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:48.682666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77067-b275-408d-b5f0-a6a9b00ff2e7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:48.683118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77067-b275-408d-b5f0-a6a9b00ff2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:48.683340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77067-b275-408d-b5f0-a6a9b00ff2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:48.775292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-67b77067-b275-408d-b5f0-a6a9b00ff2e7 None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:48.777842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67b77067-b275-408d-b5f0-a6a9b00ff2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:48.855135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:48.894189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 673/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:16:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:50.477836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a50b-0d3d-4b03-b522-e48464187804 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:50.478381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a50b-0d3d-4b03-b522-e48464187804 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:50.478381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a50b-0d3d-4b03-b522-e48464187804 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:50.480644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 681/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:50.486437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608bc95-7fa7-4c4d-a975-1bd0b6aef29a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:50.487755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608bc95-7fa7-4c4d-a975-1bd0b6aef29a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:50.487755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608bc95-7fa7-4c4d-a975-1bd0b6aef29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:50.570733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2608bc95-7fa7-4c4d-a975-1bd0b6aef29a None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:50.571858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2608bc95-7fa7-4c4d-a975-1bd0b6aef29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:50.686408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 674/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:16:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:54.273860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78588-eb64-4533-a685-28df6f7e35f6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:54.274422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78588-eb64-4533-a685-28df6f7e35f6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:54.274422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78588-eb64-4533-a685-28df6f7e35f6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:54.275628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 682/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:54.281764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4bf3c-4be4-453a-8109-8038ff21303c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:54.282231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4bf3c-4be4-453a-8109-8038ff21303c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:54.282328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4bf3c-4be4-453a-8109-8038ff21303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:54.331139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:54.358496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-01b4bf3c-4be4-453a-8109-8038ff21303c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:54.359703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01b4bf3c-4be4-453a-8109-8038ff21303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:54.472912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 675/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:54.775702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:54.775969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:54.776706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:54.778196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 683/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:54.786368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55be57-5777-4c0f-bda1-e0a4449f2810 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:54.787057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55be57-5777-4c0f-bda1-e0a4449f2810 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:54.787696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55be57-5777-4c0f-bda1-e0a4449f2810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:54.882661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1e55be57-5777-4c0f-bda1-e0a4449f2810 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:54.884140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e55be57-5777-4c0f-bda1-e0a4449f2810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:54.937328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:55.011642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 676/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:16:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:16:55.019611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:55.021494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:55.190230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd6ab1-bd1a-4c5d-bad1-8622208bff8a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:55.269893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:55.313507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:55.315283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:55.315622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:55.316040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:55.317695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:55.422687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:55.424366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927e80f-7f55-447a-bcf7-f5e801f1072b service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:55.443841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 684/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:16:57.874991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:57.875615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:57.875615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:57.877026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 677/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:16:57.882279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118836-e397-47b6-b109-b537eb69787d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:57.882715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118836-e397-47b6-b109-b537eb69787d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:57.882965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118836-e397-47b6-b109-b537eb69787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:57.956493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8c118836-e397-47b6-b109-b537eb69787d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:16:57.957818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8c118836-e397-47b6-b109-b537eb69787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:57.997896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:58.068221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 685/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:16:58.367636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:16:58.369281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:58.550925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcc7f2-85ee-4bdc-8d96-69f6e3d70965 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:16:58.561854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:16:58.672605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:16:58.673251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:16:58.673524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:16:58.673786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:16:58.675433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:16:58.773018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:16:58.774618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f599daa-6f1f-4490-8da0-a9c8e2299644 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:16:58.793869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 678/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:17:01.333664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944feea2-0e99-4eb2-8afa-912c540f2c02 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:01.334195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944feea2-0e99-4eb2-8afa-912c540f2c02 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:01.334195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944feea2-0e99-4eb2-8afa-912c540f2c02 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:01.335892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 686/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:01.342047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd37c2-742e-4ad5-acd3-88bb0ce4c604 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:01.342455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd37c2-742e-4ad5-acd3-88bb0ce4c604 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:01.342642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd37c2-742e-4ad5-acd3-88bb0ce4c604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:01.359359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:01.414987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-57bd37c2-742e-4ad5-acd3-88bb0ce4c604 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:01.416128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-57bd37c2-742e-4ad5-acd3-88bb0ce4c604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:01.523323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 679/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:01.816014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:01.816373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:01.816719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:01.818146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 687/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:01.824447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab1deb-b2c4-40a5-8133-237f9c85d2e6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:01.824833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab1deb-b2c4-40a5-8133-237f9c85d2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:01.825341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab1deb-b2c4-40a5-8133-237f9c85d2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:01.905506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8cab1deb-b2c4-40a5-8133-237f9c85d2e6 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:01.906849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8cab1deb-b2c4-40a5-8133-237f9c85d2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:01.943182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:02.035917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 680/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:17:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:02.045004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:02.046533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:02.205726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9467-560e-4488-a829-3ef0193e6af8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:02.229650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:02.328840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:02.329137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:02.329366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:02.329629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:02.331218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:02.431563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:02.433825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8f2e5-727f-4202-9ddf-62acd1c08751 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:02.452678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 688/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:02.480072 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0471ac-e9d3-4e15-8d56-312861ed1fa9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:02.480609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0471ac-e9d3-4e15-8d56-312861ed1fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:02.480609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0471ac-e9d3-4e15-8d56-312861ed1fa9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:02.482110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 681/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:02.487468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e8c2-c24c-4659-9ab5-61b5031b05cf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:02.487947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e8c2-c24c-4659-9ab5-61b5031b05cf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:02.488381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e8c2-c24c-4659-9ab5-61b5031b05cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:02.524479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:02.565626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-32c0e8c2-c24c-4659-9ab5-61b5031b05cf None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:02.567173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-32c0e8c2-c24c-4659-9ab5-61b5031b05cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:02.696985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 689/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:03.037685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:03.037965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:03.038236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:03.039689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 682/1371] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 07:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:03.044881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a40b6-1837-412c-942e-a742cd02856f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:03.045189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a40b6-1837-412c-942e-a742cd02856f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:03.045461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a40b6-1837-412c-942e-a742cd02856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:03.125356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-646a40b6-1837-412c-942e-a742cd02856f None None] MFA Rules not processed for user `ce964ecd12bd4e3d9d85bf5dc227d35f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:03.126718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-646a40b6-1837-412c-942e-a742cd02856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:03.180582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:03.214518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 690/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 07:17:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:03.223318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:03.225115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:03.348071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fac2d-f980-4a17-9f26-21ba7ceae8a9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:03.402173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:03.443394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:03.444017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:03.444384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:03.444546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:03.446413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:03.565143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:03.566591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38cbc8e-4393-4bfb-876c-1bdeef2c9ea9 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:03.592017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 683/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:04.732173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:04.732668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:04.732777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:04.734091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 691/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:04.739543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0924-2a36-4693-ad11-ccd63b878eab None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:04.739981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0924-2a36-4693-ad11-ccd63b878eab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:04.740224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0924-2a36-4693-ad11-ccd63b878eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:04.806014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e39a0924-2a36-4693-ad11-ccd63b878eab None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:04.807694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e39a0924-2a36-4693-ad11-ccd63b878eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:04.828471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:04.904837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 684/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:05.202827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:05.204410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:05.362153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13677b06-7340-4c55-acf9-f9b757c6b058 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:05.414922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:05.482763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:05.483464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:05.483805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:05.484070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:05.485691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:05.585210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:05.586966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3ccdd-c009-4be2-997e-5d2a3163ca13 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:05.606357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 692/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:07.811608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:07.812146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:07.812146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:07.813895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 685/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:07.818919 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88cadb-c686-4ada-bdf0-d77420352dad None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:07.819314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88cadb-c686-4ada-bdf0-d77420352dad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:07.819458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88cadb-c686-4ada-bdf0-d77420352dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:07.889822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3b88cadb-c686-4ada-bdf0-d77420352dad None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:07.891314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b88cadb-c686-4ada-bdf0-d77420352dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:07.898748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:08.005950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 693/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:08.294922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:08.296518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:08.464807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af12df3-4c61-4d3a-9e2f-d685703f50f6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:08.468858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:08.583804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:08.584491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:08.584865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:08.585273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:08.586965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:08.680968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:08.683468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506db1a-5965-453b-91a9-b014acedf726 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:08.710419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 686/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:10.964304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:10.964784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:10.964883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:10.966413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 694/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:10.971526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17aa06-7e4f-4b3a-bb7d-16c1cca49796 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:10.971937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17aa06-7e4f-4b3a-bb7d-16c1cca49796 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:10.972100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17aa06-7e4f-4b3a-bb7d-16c1cca49796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:10.983639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:11.046522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4c17aa06-7e4f-4b3a-bb7d-16c1cca49796 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:11.047654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4c17aa06-7e4f-4b3a-bb7d-16c1cca49796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:11.192303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 687/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:17:11.533344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:11.535317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:11.686352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:11.698317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816edf-385f-4e8f-b1e2-a59d9a3cb6ee None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:11.816544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:11.817168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:11.817441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:11.817741 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:11.819281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:11.915146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:11.917457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a89f1-a05d-451f-91b8-3bb158e0285d service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:11.927839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:11.936878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 695/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:14.741023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:14.741563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:14.741783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:14.743488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 688/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:14.750408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20f9f-a1ee-4ca3-b93c-270e36737455 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:14.750835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20f9f-a1ee-4ca3-b93c-270e36737455 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:14.750953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20f9f-a1ee-4ca3-b93c-270e36737455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:14.818064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fcd20f9f-a1ee-4ca3-b93c-270e36737455 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:14.819260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fcd20f9f-a1ee-4ca3-b93c-270e36737455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:14.927758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 696/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:15.296418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:15.298087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:15.395191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:15.465583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f383503-2ed8-4c05-b3b2-fbc69636c721 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:15.594307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:15.594917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:15.595198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:15.595471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:15.597124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:15.649013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:15.695419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:15.698013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89c74b-2306-47c8-806c-3901afdc8a7a service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:15.718172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 689/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:17.940401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:17.941043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:17.941043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:17.942322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 697/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:17.948456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daef7de-5e3d-4937-adb6-51be78a37b96 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:17.948883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daef7de-5e3d-4937-adb6-51be78a37b96 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:17.949093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daef7de-5e3d-4937-adb6-51be78a37b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:18.016404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0daef7de-5e3d-4937-adb6-51be78a37b96 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:18.017554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0daef7de-5e3d-4937-adb6-51be78a37b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:18.121156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 690/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:18.418143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:18.419657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:18.495568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:18.566177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62aeb3-219e-4ce5-9a94-a90cbd202eb6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:18.676921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:18.677485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:18.677752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:18.678012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:18.679581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:18.721849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:18.773615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:18.775352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e910a-ec02-44e1-9e4f-a95bd0d22275 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:18.795645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 698/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:18.913406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fec3ba-9e77-4236-bc4d-a6df7b7326ba None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:18.913724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fec3ba-9e77-4236-bc4d-a6df7b7326ba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:18.914019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fec3ba-9e77-4236-bc4d-a6df7b7326ba None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:18.915386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 691/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:18.922582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e019fb9-8f89-4094-8be6-acc1e6812773 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:18.922874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e019fb9-8f89-4094-8be6-acc1e6812773 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:18.923182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e019fb9-8f89-4094-8be6-acc1e6812773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:18.998354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7e019fb9-8f89-4094-8be6-acc1e6812773 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:18.999477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e019fb9-8f89-4094-8be6-acc1e6812773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:19.099704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 699/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:19.397887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:19.398366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:19.398471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:19.399729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 692/1391] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 07:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:19.405820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74091-9b2f-4ddf-9e15-140c8fc00d14 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:19.406130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74091-9b2f-4ddf-9e15-140c8fc00d14 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:19.406389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74091-9b2f-4ddf-9e15-140c8fc00d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:19.423443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:19.485070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-87b74091-9b2f-4ddf-9e15-140c8fc00d14 None None] MFA Rules not processed for user `20ca1a72134341ed9dbe8b58c339873e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:19.486428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87b74091-9b2f-4ddf-9e15-140c8fc00d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:19.562546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 700/1392] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 07:17:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:19.572511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:19.574069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:19.605804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:19.679039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce229ef2-57be-439d-9f0a-5bcb63d7bfd2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:19.759031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:19.760123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:19.760404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:19.760671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:19.762320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:19.839876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:19.872584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:19.874040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fc6ca-068d-4010-b6f8-ad98bffb6d46 service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:19.892828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 693/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:20.630634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85951-4467-48b9-8946-3108c5cf8a38 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:20.631158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85951-4467-48b9-8946-3108c5cf8a38 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:20.631374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85951-4467-48b9-8946-3108c5cf8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:20.704446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f6c85951-4467-48b9-8946-3108c5cf8a38 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:20.705718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6c85951-4467-48b9-8946-3108c5cf8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:20.762046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:20.793934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 701/1394] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:17:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:20.837133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d145-8e33-4522-997a-f46781d4c2ff None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:20.837674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d145-8e33-4522-997a-f46781d4c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:20.837951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d145-8e33-4522-997a-f46781d4c2ff None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:20.839384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 694/1395] 10.222.0.52 () {54 vars in 943 bytes} [Wed Sep 28 07:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:20.845785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:20.846075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:20.846351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:20.934008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] MFA Rules not processed for user `c4389bd42da64dccbf958503dccf87e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:20.935221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:21.011426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a473-5fec-4212-aef8-003dc0c46186 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:21.011699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a473-5fec-4212-aef8-003dc0c46186 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:21.011981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a473-5fec-4212-aef8-003dc0c46186 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:21.013332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 695/1396] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:17:21.018386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:21.018663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:21.018942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:21.041654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 702/1397] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Sep 28 07:17:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:17:21.051528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:21.061648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c23f-81e4-4253-bf04-85bf9b0c30d1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:21.088115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:21.091045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:21.092568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:21.171393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:21.172002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:21.172279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:21.172550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:21.174255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:21.190982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 696/1398] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:17:21.270642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:21.272228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06b9c5-f9e1-4b58-8e3f-b59b3f320db2 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:21.280324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 703/1399] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 07:17:21.484771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:21.486127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:21.531246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:21.617274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7797b84-26ea-4521-8d95-86e9bd50fd8e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:21.761455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:21.762217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:21.762501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:21.762772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:21.764447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:21.802249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:21.865055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:21.866684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb326a-f51a-4277-a8e6-ceeee9b91af7 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:21.888328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 697/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:24.131681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:24.132295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:24.132295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:24.134775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 704/1401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:24.139497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b169-9e16-4f23-b6d7-eeee0a8eeaa9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:24.139918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b169-9e16-4f23-b6d7-eeee0a8eeaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:24.140038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b169-9e16-4f23-b6d7-eeee0a8eeaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:24.213779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dbf2b169-9e16-4f23-b6d7-eeee0a8eeaa9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:24.216599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dbf2b169-9e16-4f23-b6d7-eeee0a8eeaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:24.337751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 698/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:24.624395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:24.625940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:24.694824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:24.799295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93228e0b-362f-4dbd-8e84-b2aa67a8ce9e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:24.924053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:24.924656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:24.924933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:24.925231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:24.926862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:24.947407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:25.023760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:25.025688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e96c1-ce66-46cc-9232-1b326ec60042 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:25.045389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 705/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:27.307318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:27.307908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:27.307908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:27.309392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 699/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:27.314993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621d93-6915-4738-b700-c9c3b0746035 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:27.315561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621d93-6915-4738-b700-c9c3b0746035 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:27.315561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621d93-6915-4738-b700-c9c3b0746035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:27.389692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-89621d93-6915-4738-b700-c9c3b0746035 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:27.391077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-89621d93-6915-4738-b700-c9c3b0746035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:27.490284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:27.496794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 706/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:27.784255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:27.785840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:27.796148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:27.949112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69117085-4cf1-4ef7-ad97-97dd6e2ab23a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:28.038301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:28.075677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:28.076296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:28.076550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:28.076815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:28.078420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:28.176852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:28.177621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad86cf-c409-48a9-9225-21dd2a37c994 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:28.198974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 700/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:30.406274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:30.406846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:30.406846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:30.408498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 707/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:30.413675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb002-9d64-428d-b45b-6258c4cc2d98 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:30.414070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb002-9d64-428d-b45b-6258c4cc2d98 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:30.414230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb002-9d64-428d-b45b-6258c4cc2d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:30.488745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3abeb002-9d64-428d-b45b-6258c4cc2d98 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:30.489883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3abeb002-9d64-428d-b45b-6258c4cc2d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:30.547242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:30.606700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 701/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:30.897310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:30.900608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:31.076686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a21f7b-04c5-46c0-88e4-d71687a735a8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:31.155656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:31.208336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:31.209041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:31.209326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:31.209597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:31.211223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:31.316408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:31.318024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0aa93-28c8-4b9b-98df-f93218a057e9 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:31.338402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 708/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:40.605380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=23 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:41.335235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=64 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:44.270710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b14fd7-5796-45f0-a0bc-a6f6e278968f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:44.271250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b14fd7-5796-45f0-a0bc-a6f6e278968f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:44.271365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b14fd7-5796-45f0-a0bc-a6f6e278968f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:44.272622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 702/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 07:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:44.278398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e8f-8698-4883-b99c-89a6ca98e69b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:44.278832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e8f-8698-4883-b99c-89a6ca98e69b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:44.278945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e8f-8698-4883-b99c-89a6ca98e69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:44.353225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a0622e8f-8698-4883-b99c-89a6ca98e69b None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:44.354346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a0622e8f-8698-4883-b99c-89a6ca98e69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:44.461333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 709/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 07:17:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:44.475505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c243d-1ff6-427c-afae-1cc36d29960b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:44.475786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c243d-1ff6-427c-afae-1cc36d29960b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:44.476362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c243d-1ff6-427c-afae-1cc36d29960b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:44.477539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 703/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 07:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:44.482080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf67-bb81-4d47-8eb4-aebb40ce47d4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:44.482369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf67-bb81-4d47-8eb4-aebb40ce47d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:44.482646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf67-bb81-4d47-8eb4-aebb40ce47d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:44.554800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:44.558085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-71b6cf67-bb81-4d47-8eb4-aebb40ce47d4 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:44.559838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-71b6cf67-bb81-4d47-8eb4-aebb40ce47d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:44.791524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 710/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 07:17:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:17:47.355181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:47.355770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:47.355770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:47.357167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:47.362506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da3d6-e8c1-4008-9cf2-4480dcfbde2f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:47.362926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da3d6-e8c1-4008-9cf2-4480dcfbde2f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:47.363050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da3d6-e8c1-4008-9cf2-4480dcfbde2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:47.428089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fe6da3d6-e8c1-4008-9cf2-4480dcfbde2f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:47.429523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fe6da3d6-e8c1-4008-9cf2-4480dcfbde2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:47.483423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:47.524384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:47.820848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:47.822568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:47.966353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be52b2-1b1f-4728-9cfa-3511dfb3534e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:48.037903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:48.079194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:48.079791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:48.080064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:48.080330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:48.081960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:48.169699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:48.171265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23426e51-8829-4ff0-88ac-8487a3251736 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:48.189247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 705/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:49.330753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:49.331329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:49.331329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:49.332641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:49.338835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923136df-43a8-4c7d-9ee9-34e162afbd2c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:49.339313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923136df-43a8-4c7d-9ee9-34e162afbd2c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:49.339416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923136df-43a8-4c7d-9ee9-34e162afbd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:49.413365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-923136df-43a8-4c7d-9ee9-34e162afbd2c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:49.414597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-923136df-43a8-4c7d-9ee9-34e162afbd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:49.441222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:49.511301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:49.660299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:49.661768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:49.809683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0122dcc-3e8c-4605-9a4c-537438fdef1e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:49.849676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:49.923947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:49.924437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:49.924706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:49.925007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:49.926576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:50.014727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:50.016294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1c272-f482-4875-a4b8-546cf315bd13 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:50.033473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 713/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:51.170882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:51.171466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:51.171466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:51.172749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:51.179566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3bee-264f-47d2-b523-97baf8461eab None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:51.180025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3bee-264f-47d2-b523-97baf8461eab None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:51.180121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3bee-264f-47d2-b523-97baf8461eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:51.233881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:51.255032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3daf3bee-264f-47d2-b523-97baf8461eab None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:51.256214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3daf3bee-264f-47d2-b523-97baf8461eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:51.350481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:51.501017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:51.502455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:51.650963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084858e-d8fd-4eed-9e31-f552d939bfb6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:51.662970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:51.765011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:51.765548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:51.765846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:51.766111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:51.767740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:51.865836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:51.868933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b894d1-9c63-4c85-914f-32546984deb3 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:51.891571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 708/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:53.041198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:53.041726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:53.041726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:53.043098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:53.049438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a195-2d6c-49ab-9dd3-d54a3bc621e9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:53.049912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a195-2d6c-49ab-9dd3-d54a3bc621e9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:53.050019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a195-2d6c-49ab-9dd3-d54a3bc621e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:53.071025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:53.129682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5b87a195-2d6c-49ab-9dd3-d54a3bc621e9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:53.131215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5b87a195-2d6c-49ab-9dd3-d54a3bc621e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:53.229052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:53.383249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:53.384755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:53.492337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:53.533328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813207bb-423a-401b-a5f4-71abe6510deb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:53.646465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:53.646980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:53.647812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:53.648129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:53.649772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:53.724586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:53.738868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:53.740443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d7451-758a-4b50-a2e9-7b3029d5b30b service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:53.758009 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 716/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:54.887674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:54.888252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:54.888252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:54.889573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:54.895755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218568-7781-4d59-9922-5245a8aed7ba None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:54.896238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218568-7781-4d59-9922-5245a8aed7ba None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:54.896353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218568-7781-4d59-9922-5245a8aed7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:54.971719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-63218568-7781-4d59-9922-5245a8aed7ba None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:54.972838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-63218568-7781-4d59-9922-5245a8aed7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:55.068128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:55.218973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:55.220540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:55.302973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:55.365636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59e06-a716-4325-920b-63065d6eb3c4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:55.477482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:55.478084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:55.478360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:55.478628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:55.480210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:55.531776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:55.574195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:55.575796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9064a-2107-4809-87aa-606f4fd25574 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:55.593732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 711/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:56.680429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:56.681017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:56.681017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:56.682478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:17:56.691942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc77ad0-3881-4b50-9afc-7e1b857129a4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:56.691942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc77ad0-3881-4b50-9afc-7e1b857129a4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:56.691942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc77ad0-3881-4b50-9afc-7e1b857129a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:56.767767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3dc77ad0-3881-4b50-9afc-7e1b857129a4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:56.768991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3dc77ad0-3881-4b50-9afc-7e1b857129a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:56.863607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:57.014744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:57.016175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:57.065449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:57.160265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60220b6f-9978-43eb-837f-f1e3c401ab54 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:57.273989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:57.274536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:57.274808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:57.275077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:57.276675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:57.283505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:57.368333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:57.369930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7de4e9-1867-48d2-88c3-8c9855b4be05 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:57.387244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 719/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:17:58.445773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:58.446305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:58.446305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:58.447635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:17:58.453793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbdba-2f37-4dd0-9e87-901478c58cd6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:58.454251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbdba-2f37-4dd0-9e87-901478c58cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:58.454361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbdba-2f37-4dd0-9e87-901478c58cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:58.528460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-141cbdba-2f37-4dd0-9e87-901478c58cd6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:17:58.529657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-141cbdba-2f37-4dd0-9e87-901478c58cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:58.595761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:58.626029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:17:58.778948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:17:58.780460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:58.798419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:58.925622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bac52-3890-4350-a159-53552732f8c7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:17:59.012225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:17:59.038508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:17:59.039100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:17:59.039373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:17:59.039641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:17:59.041247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:17:59.128802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:17:59.130427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb35f5c-e542-463a-8360-d7b46b35ae4d service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:17:59.149068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 714/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:00.199542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:00.200112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:00.200112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:00.201438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:00.207596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e6e7-5d8d-4938-a01f-e155675d5c63 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:00.208013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e6e7-5d8d-4938-a01f-e155675d5c63 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:00.208143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e6e7-5d8d-4938-a01f-e155675d5c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:00.282956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b8b8e6e7-5d8d-4938-a01f-e155675d5c63 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:00.284161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b8b8e6e7-5d8d-4938-a01f-e155675d5c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:00.326090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:00.384303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:00.541229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:00.542732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:00.687135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3988ef-f02d-4cc4-ac3a-4cbb891a9aa6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:00.741392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:00.798192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:00.798783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:00.799061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:00.799324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:00.800887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:00.892841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:00.894473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e59b7-c0b4-465d-b074-a6874bdd41a2 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:00.912002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 722/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:01.959883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:01.960405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:01.960405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:01.961700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:01.967844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a210b-940f-40e5-9e60-a6ae50054eac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:01.968283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a210b-940f-40e5-9e60-a6ae50054eac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:01.968395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a210b-940f-40e5-9e60-a6ae50054eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:02.043937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bd1a210b-940f-40e5-9e60-a6ae50054eac None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:02.045149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bd1a210b-940f-40e5-9e60-a6ae50054eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:02.047121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:02.141449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:02.290067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:02.291596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:02.437245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9bcb3-200d-4a16-9852-5d6303b25f8b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:02.464647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:02.549412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:02.549932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:02.550203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:02.550465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:02.552036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:02.638773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:02.640321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f493b-a79e-46f1-b554-e61184a3cc1b service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:02.658141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 717/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:04.720222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:04.720750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:04.720750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:04.722142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:04.727641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918908-df22-42d0-aafd-0bb3ed088c2c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:04.728108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918908-df22-42d0-aafd-0bb3ed088c2c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:04.728211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918908-df22-42d0-aafd-0bb3ed088c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:04.768536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:04.794441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f0918908-df22-42d0-aafd-0bb3ed088c2c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:04.795658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f0918908-df22-42d0-aafd-0bb3ed088c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:04.891896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:05.184533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:05.186369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:05.291685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caf27c-e520-41e4-9e28-672f6cfac580 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:05.325460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:05.366831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:05.367428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:05.367696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:05.367952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:05.369550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:05.473728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:05.475585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ebd5c-e50f-40be-a098-60a9fea4d620 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:05.493671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 725/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:07.808349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:07.808906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:07.808906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:07.810287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:07.815743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedcb6a-39d8-4da4-83e3-4f123d1ded8d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:07.816155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedcb6a-39d8-4da4-83e3-4f123d1ded8d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:07.816290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedcb6a-39d8-4da4-83e3-4f123d1ded8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:07.882220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cdedcb6a-39d8-4da4-83e3-4f123d1ded8d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:07.883421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cdedcb6a-39d8-4da4-83e3-4f123d1ded8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:07.896560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:07.979673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:08.270827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:08.273362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:08.375204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75f58-ad45-497f-ba26-2816698d49cd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:08.388995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:08.449697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:08.450297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:08.450569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:08.450833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:08.452400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:08.559110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:08.560828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea7dae-e4dd-40ae-a678-d9767350e219 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:08.579515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 720/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:11.044371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:11.044920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:11.044920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:11.046378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:11.051748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ceab-c519-45c7-9ede-5dfab7b6460f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:11.052217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ceab-c519-45c7-9ede-5dfab7b6460f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:11.052375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ceab-c519-45c7-9ede-5dfab7b6460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:11.100831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:11.118863 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6842ceab-c519-45c7-9ede-5dfab7b6460f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:11.120061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6842ceab-c519-45c7-9ede-5dfab7b6460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:11.217202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:11.503497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:11.505053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:11.606813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcd90a-7328-44f9-a2f2-f80a48a872a3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:11.648366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:11.694170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:11.695480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:11.695867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:11.696134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:11.697765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:11.805615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:11.807039 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2111d-e56f-4545-b183-d4cb6b8836b4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:11.825721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 728/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:14.161222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:14.161845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:14.161845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:14.163614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:14.169033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa7a2-55a7-4413-ae5a-0260b3de9a39 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:14.169451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa7a2-55a7-4413-ae5a-0260b3de9a39 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:14.169556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa7a2-55a7-4413-ae5a-0260b3de9a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:14.233753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fd0aa7a2-55a7-4413-ae5a-0260b3de9a39 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:14.234935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd0aa7a2-55a7-4413-ae5a-0260b3de9a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:14.254483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:14.331486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:14.615384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:14.617466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:14.720510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7c68b-c7c4-4d09-bd7a-d977aa615598 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:14.741960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:14.795803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:14.796402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:14.796677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:14.796962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:14.798562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:14.906462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:14.907788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4539b6-3c69-40ee-8cff-21f61cc85c71 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:14.925726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 723/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:17.117579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:17.118172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:17.118172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:17.119442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:17.124759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ef43a-3971-4db4-8a4e-a637ae18420a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:17.125192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ef43a-3971-4db4-8a4e-a637ae18420a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:17.125375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ef43a-3971-4db4-8a4e-a637ae18420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:17.188381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:17.191303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e11ef43a-3971-4db4-8a4e-a637ae18420a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:17.193081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e11ef43a-3971-4db4-8a4e-a637ae18420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:17.288531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:17.575337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:17.577030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:17.677357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3b9f-6a4c-45ee-bf36-d37986c183c4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:17.728827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:17.753969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:17.754565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:17.754840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:17.755110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:17.757532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:17.864071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:17.865509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dffa73-c009-4cf1-8acf-9cec3b939847 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:17.884590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 731/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:20.099902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:20.101076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:20.101225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:20.102613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:20.108862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0093c-9321-4d53-93d7-8a65e309c6c1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:20.109371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0093c-9321-4d53-93d7-8a65e309c6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:20.109479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0093c-9321-4d53-93d7-8a65e309c6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:20.175318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8eb0093c-9321-4d53-93d7-8a65e309c6c1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:20.176564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8eb0093c-9321-4d53-93d7-8a65e309c6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:20.206199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:20.274379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:20.559192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:20.561016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:20.664193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f8545-968c-48d9-9003-d2fbb0804a99 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:20.692392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:20.738373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:20.738973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:20.739244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:20.739514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:20.741456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:20.850496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:20.852116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b348d1-a235-4406-84ff-8db66d013386 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:20.870206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 726/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:23.065296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:23.065861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:23.065861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:23.067253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:23.072702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4509e3-8083-441c-8a2f-f324a9b6eaea None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:23.073147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4509e3-8083-441c-8a2f-f324a9b6eaea None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:23.073275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4509e3-8083-441c-8a2f-f324a9b6eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:23.138980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fb4509e3-8083-441c-8a2f-f324a9b6eaea None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:23.140194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fb4509e3-8083-441c-8a2f-f324a9b6eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:23.149176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:23.235274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:23.515294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:23.516846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:23.618320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5759f-5d15-4a7d-8d96-5dd35e5d87d6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:23.671469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:23.690415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:23.691014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:23.691285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:23.691658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:23.693279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:23.802291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:23.804312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31293-432d-431a-a107-1d697a21cf8d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:23.822868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 734/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:26.078973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:26.079538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:26.079538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:26.080982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:26.086709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a819-b87f-4fe6-8aa2-c11b14468f87 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:26.087145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a819-b87f-4fe6-8aa2-c11b14468f87 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:26.087261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a819-b87f-4fe6-8aa2-c11b14468f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:26.152867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5a33a819-b87f-4fe6-8aa2-c11b14468f87 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:26.154086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5a33a819-b87f-4fe6-8aa2-c11b14468f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:26.193514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:26.250770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:18:26.541947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:26.543513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:26.646684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0c2bb-ce8b-4de2-9f68-e2470126a95a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:26.684075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:26.726356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:26.726959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:26.727230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:26.727495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:26.729108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:26.837152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:26.838569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941e66f-6079-48e4-9297-91e72a8b66ef service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:26.856778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 729/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:29.034444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:29.034989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:29.034989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:29.036321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:29.041694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b4f5-b93f-4cd8-b74f-e6402968e59b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:29.042128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b4f5-b93f-4cd8-b74f-e6402968e59b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:29.042241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b4f5-b93f-4cd8-b74f-e6402968e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:29.107630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-57e5b4f5-b93f-4cd8-b74f-e6402968e59b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:29.108879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-57e5b4f5-b93f-4cd8-b74f-e6402968e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:29.127362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:29.205119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:29.492724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:29.494355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:29.595803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45caf3-adc6-42f9-a711-05c58c9d44ad None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:29.660924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:29.667956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:29.668571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:29.668847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:29.669142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:29.670729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:29.779721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:29.781183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274dc52-1741-4542-b0f9-2a857194e408 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:29.798557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 737/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:31.985599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25080a6-e059-4055-8542-dee194522c27 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:31.986129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25080a6-e059-4055-8542-dee194522c27 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:31.986129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25080a6-e059-4055-8542-dee194522c27 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:31.987383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:31.993208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19587d1e-58da-4f0f-9351-4c98b62193f7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:31.993650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19587d1e-58da-4f0f-9351-4c98b62193f7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:31.993767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19587d1e-58da-4f0f-9351-4c98b62193f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:32.058877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-19587d1e-58da-4f0f-9351-4c98b62193f7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:32.060329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-19587d1e-58da-4f0f-9351-4c98b62193f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:32.108597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:32.157450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:32.444520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25080a6-e059-4055-8542-dee194522c27 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:32.446086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d25080a6-e059-4055-8542-dee194522c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:32.548207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25080a6-e059-4055-8542-dee194522c27 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:32.595184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:32.621218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:32.621818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:32.622094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:32.622362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:32.623951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:32.733638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:32.735089 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4bcd5-7e29-4b9c-81e7-e8d5aa302e3b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:32.753084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 732/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:42.155875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=22 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:42.750810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=66 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:52.655231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724f4a7-0c28-45db-b551-ad8af0f75120 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:52.655784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724f4a7-0c28-45db-b551-ad8af0f75120 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:52.655784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724f4a7-0c28-45db-b551-ad8af0f75120 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:52.657049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 739/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:52.662905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fb91-3209-41ae-926b-73042e27903f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:52.663351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fb91-3209-41ae-926b-73042e27903f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:52.663456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fb91-3209-41ae-926b-73042e27903f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:52.731850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5494fb91-3209-41ae-926b-73042e27903f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:52.733068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5494fb91-3209-41ae-926b-73042e27903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:52.828225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 733/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:53.127316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:53.127629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:53.127899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:53.129848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 740/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 07:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:53.135126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a373a-f760-405b-9693-a5c8805c6c96 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:53.135411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a373a-f760-405b-9693-a5c8805c6c96 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:53.135681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a373a-f760-405b-9693-a5c8805c6c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:53.210718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-270a373a-f760-405b-9693-a5c8805c6c96 None None] MFA Rules not processed for user `20ca1a72134341ed9dbe8b58c339873e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:53.211905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-270a373a-f760-405b-9693-a5c8805c6c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:53.231713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:53.286034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 734/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 07:18:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:53.293657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:53.295233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:53.396978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d6b38-ddc9-4eb0-88ef-31598c74b1fd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:53.469088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:53.469680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:53.469951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:53.470215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:53.471780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:53.525780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:53.580606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:53.582606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631081da-d1ba-4770-81a2-a7dc36639586 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:53.600886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 741/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:18:55.854278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:55.854815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:55.854815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:55.856185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 735/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:55.861371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af945e5d-4ce5-4717-b808-14e34d686aee None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:55.861783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af945e5d-4ce5-4717-b808-14e34d686aee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:55.861910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af945e5d-4ce5-4717-b808-14e34d686aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:55.926505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-af945e5d-4ce5-4717-b808-14e34d686aee None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:55.927685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-af945e5d-4ce5-4717-b808-14e34d686aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:56.022185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 742/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:18:56.296321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:56.297942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:56.423500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e96c-bfaa-4656-9ed2-ef32406171a5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:56.589905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:56.635156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:56.635755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:56.636029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:56.636295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:56.636923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:56.638675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:18:56.638675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:18:56.638675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:18:56.639799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bf6f4-8e42-4f34-bf42-69b04fa11901 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:56.658757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 736/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 07:18:56 2022] GET /identity/v3/projects => generated 2553 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:18:58.880707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:58.881288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:58.881288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:58.882626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 743/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:18:58.888010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87713a6-ae0f-4d96-8fed-ce3c7125f3e9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:58.888464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87713a6-ae0f-4d96-8fed-ce3c7125f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:58.888560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87713a6-ae0f-4d96-8fed-ce3c7125f3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:58.954088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a87713a6-ae0f-4d96-8fed-ce3c7125f3e9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:18:58.955195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a87713a6-ae0f-4d96-8fed-ce3c7125f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:59.047919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 737/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:18:59.342611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:18:59.344259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:59.354810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:59.491332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa989-6bfc-42ea-8f34-9561808430f2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:18:59.569125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:18:59.601851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:18:59.602382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:18:59.602656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:18:59.602920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:18:59.604525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:18:59.694048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:18:59.695675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cb744-dd1a-47bf-ad6a-8e078941de71 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:18:59.714298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 744/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:19:02.083038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:02.083581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:02.083581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:02.084904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 738/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:19:02.090142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635866-c729-4ea5-a56c-58c1ca4876c8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:02.090571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635866-c729-4ea5-a56c-58c1ca4876c8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:02.090708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635866-c729-4ea5-a56c-58c1ca4876c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:02.157489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-41635866-c729-4ea5-a56c-58c1ca4876c8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:19:02.158685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-41635866-c729-4ea5-a56c-58c1ca4876c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:02.204888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:02.256952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 745/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:19:02.551012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:19:02.552563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:02.575911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:02.654796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f3a38-6843-49cd-a929-5b9f88a6659b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:19:02.730435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:19:02.731033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:02.731304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:02.731566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:02.733170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:02.806315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:02.841266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:19:02.842682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c3255-5c5c-419f-bbec-92b6920db5c1 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:19:02.861580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 739/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:19:05.064556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:05.065218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:05.065218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:05.066671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 746/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:19:05.072070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa30fff-9ec7-4869-8d2f-860586c4a8a5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:05.072506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa30fff-9ec7-4869-8d2f-860586c4a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:05.072609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa30fff-9ec7-4869-8d2f-860586c4a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:05.138615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8aa30fff-9ec7-4869-8d2f-860586c4a8a5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:19:05.139812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8aa30fff-9ec7-4869-8d2f-860586c4a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:05.233734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 740/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:19:05.520970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:19:05.522501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:05.625471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457a663-e457-4c1a-ba1d-6449e474b3da None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:19:05.698564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:05.700104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:19:05.700795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:05.701712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:05.701712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:05.703679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:05.814924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:19:05.817386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3e972-d1bd-46f9-aa82-94eca5ce0506 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:19:05.835171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 747/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:19:08.144721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:08.145327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:08.145327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:08.146614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 741/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:19:08.152031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce983b7-7957-4e1b-8a12-fe6cd019c809 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:08.152465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce983b7-7957-4e1b-8a12-fe6cd019c809 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:08.152579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce983b7-7957-4e1b-8a12-fe6cd019c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:08.217228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0ce983b7-7957-4e1b-8a12-fe6cd019c809 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:19:08.218415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ce983b7-7957-4e1b-8a12-fe6cd019c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:08.269779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:08.314124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 748/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:19:08.608261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:19:08.609836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:08.645063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:08.714036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ce7f7-bc85-4350-b919-a666f2cdc6fc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:19:08.790460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:19:08.791064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:08.791340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:08.791609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:08.793261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:08.872421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:08.903063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:19:08.904493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d03bf-0f7e-43d2-a16b-2fcceaddea7f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:19:08.922512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 742/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:19:11.267182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:11.267731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:11.267731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:11.269869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 749/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 07:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:19:11.275403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6c70-bb80-4226-b53e-be3f36ad3f28 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:11.275845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6c70-bb80-4226-b53e-be3f36ad3f28 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:11.276006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6c70-bb80-4226-b53e-be3f36ad3f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:11.345322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-184e6c70-bb80-4226-b53e-be3f36ad3f28 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:19:11.347086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-184e6c70-bb80-4226-b53e-be3f36ad3f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:11.442080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 743/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 07:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:19:11.730554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:19:11.732125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:11.834865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a9cc5-8361-4564-8ebd-4d629845c2fb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:19:11.906883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:19:11.907483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:19:11.907759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:19:11.908026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:19:11.910129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:19:11.918442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:12.021822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:19:12.023158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19db5-f5be-445c-a3a8-8365a2ec648e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:19:12.041487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 750/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:19:21.440413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=82 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:19:22.039154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:24.373619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:24.374257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:24.374367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:24.449691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:20:24.450898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:24.548333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 744/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:20:24.556612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:24.557156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:24.557269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:24.639113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:20:24.641157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:24.738015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 751/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:20:24.742558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:24.743777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:24.755412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01729a34-37d6-420b-b63a-79adceac0a6b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:24.816008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:24.849436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:24.850029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:24.850300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:24.850563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:24.851227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:20:24.851520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:24.852517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:24.852517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:20:24.852517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:20:24.853414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:24.864866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 745/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:20:24.873281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:24.882244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b531d0-202b-4033-a020-c56d1501db3c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:24.942049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:24.975489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:24.976080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:24.976353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:24.976618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:24.977269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:24.978970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:24.978970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:20:24.978970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:20:24.980146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:25.035038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 752/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:20:24 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:20:25.039266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:25.040793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:25.074090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bb405-5f34-42cd-8fcd-2064cb3f5fab admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:25.171583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:25.172180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.172452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.172716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.173378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:25.174376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:25.174376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:20:25.174376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:20:25.175579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:25.216820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 746/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:20:25 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:20:25.221415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:25.223024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:25.283986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f23622-3128-4115-b9c0-fd5325909729 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:25.368412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:25.380738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:25.381395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.381674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.381944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.382607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:25.383636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:25.383636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:20:25.383636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:20:25.384541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:25.398003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 753/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:20:25.402094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:25.403645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:25.415287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267dba-4b4a-45ad-a94c-2077b9f0c880 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:25.441732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:25.511013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:25.511534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.511804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.512064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.512685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:25.513701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:25.513701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:20:25.513701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:20:25.515241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f922676-f34e-44fa-a146-58bf15b45974 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:25.527216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 747/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:20:25.531948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:25.533501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:25.568004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335ddd7-f5ef-431a-8e3f-5476f86014bf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:25.660096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:25.660737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.661035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.661303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] PATH_INFO: `/v3/projects/5034390a05dc4ff1a46373745be0f0a5/users/ec131d3b191547a887f8ca7accb8d8a4/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.683714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5034390a05dc4ff1a46373745be0f0a5, user_id=ec131d3b191547a887f8ca7accb8d8a4, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:25.684761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:25.684761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:20:25.684761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:20:25.687216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6e2d8-3910-4431-8707-0281939f8ebc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:25.711980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 754/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:20:25 2022] PUT /identity/v3/projects/5034390a05dc4ff1a46373745be0f0a5/users/ec131d3b191547a887f8ca7accb8d8a4/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:20:25.724377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c457-7157-4a48-a879-b1fc26e0aa57 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.724665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c457-7157-4a48-a879-b1fc26e0aa57 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.724939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c457-7157-4a48-a879-b1fc26e0aa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.780616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3269c457-7157-4a48-a879-b1fc26e0aa57 None None] MFA Rules not processed for user `ec131d3b191547a887f8ca7accb8d8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:20:25.781831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3269c457-7157-4a48-a879-b1fc26e0aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:25.859894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 748/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:20:25.873045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.873333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.873852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.875174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 755/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 07:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:20:25.879914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053fa4e-6787-4cd6-aeb7-7eea64728e0a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:25.880194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053fa4e-6787-4cd6-aeb7-7eea64728e0a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:25.880461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053fa4e-6787-4cd6-aeb7-7eea64728e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:25.911694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:25.955634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6053fa4e-6787-4cd6-aeb7-7eea64728e0a None None] MFA Rules not processed for user `ce964ecd12bd4e3d9d85bf5dc227d35f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:20:25.956817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6053fa4e-6787-4cd6-aeb7-7eea64728e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.031294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 749/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:20:26.039124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:26.040667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.093249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:26.146539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ec9c-1048-4171-b06a-cf743941370b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:26.218764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:26.219355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:26.219623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:26.219890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:26.221492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.316862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:26.318227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:26.321607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:26.336965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 756/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:20:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:20:26.385040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1cbd-bb02-475e-8f25-fe426e3ed425 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:26.385326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1cbd-bb02-475e-8f25-fe426e3ed425 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:26.385597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1cbd-bb02-475e-8f25-fe426e3ed425 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:26.386829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 750/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Sep 28 07:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:20:26.390489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:26.391749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.491328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceb98d-5d10-4cd5-86b7-d52847375657 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:26.585330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:26.585932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:26.586147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:26.586413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] PATH_INFO: `/v3/users/ec131d3b191547a887f8ca7accb8d8a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:26.599927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec131d3b191547a887f8ca7accb8d8a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:26.600964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:26.600964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:20:26.600964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:20:26.602165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:26.611703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:26.667155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec131d3b191547a887f8ca7accb8d8a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:26.667683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec131d3b191547a887f8ca7accb8d8a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:26.683224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec131d3b191547a887f8ca7accb8d8a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:26.683690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec131d3b191547a887f8ca7accb8d8a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:26.712446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec131d3b191547a887f8ca7accb8d8a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:26.731972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 757/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:20:26 2022] DELETE /identity/v3/users/ec131d3b191547a887f8ca7accb8d8a4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:20:26.736597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:26.737084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:26.737194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:26.792798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:20:26.794007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.868096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:26.881090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 751/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:20:26.891268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:26.892485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:26.990890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f03d-3e0b-4cec-a4b5-daed125b793c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:27.047355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:27.068461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:27.069094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:27.069383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:27.069652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:27.071242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:27.179229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:27.180550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c086f8-af8f-44ed-8650-045c8002bcd8 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:27.198613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 758/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:20:27.553978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:20:27.555253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:20:27.567359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095fd42-0c18-4eb0-a898-999ee6d70604 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:20:27.662814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:20:27.663443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:20:27.663717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:20:27.663985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] PATH_INFO: `/v3/projects/5034390a05dc4ff1a46373745be0f0a5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:20:27.674409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5034390a05dc4ff1a46373745be0f0a5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:20:27.675425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:20:27.675425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:20:27.675425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:20:27.676615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:20:27.735445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5034390a05dc4ff1a46373745be0f0a5', 'request_id': 'req-1d03c0c6-b531-4cb3-9113-fde0e8a97123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:27.741258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5034390a05dc4ff1a46373745be0f0a5', 'request_id': 'req-1d03c0c6-b531-4cb3-9113-fde0e8a97123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:27.751097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5034390a05dc4ff1a46373745be0f0a5', 'request_id': 'req-1d03c0c6-b531-4cb3-9113-fde0e8a97123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:27.781193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] The token cache is being invalidate because project 5034390a05dc4ff1a46373745be0f0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:20:27.781482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:27.781908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1d03c0c6-b531-4cb3-9113-fde0e8a97123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:20:27.783312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 752/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:20:27 2022] DELETE /identity/v3/projects/5034390a05dc4ff1a46373745be0f0a5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:20:37.196410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=62 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:20:37.778471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:21:58.422560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:21:58.423179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:21:58.423179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:21:58.424446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:21:58.429469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35a60-4750-41b3-b008-54487a445916 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:21:58.429880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35a60-4750-41b3-b008-54487a445916 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:21:58.430021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35a60-4750-41b3-b008-54487a445916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:21:58.512520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c8c35a60-4750-41b3-b008-54487a445916 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:21:58.513753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c8c35a60-4750-41b3-b008-54487a445916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:21:58.614102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:21:58.774219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:21:58.776274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:21:58.894568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa4858-02f5-4db0-84fb-4f27ebcae6ad None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:21:58.899990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:21:58.900585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:21:58.900860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:21:58.901160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:21:58.909141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:21:58.910706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218b969-ee3d-4e55-a298-2ec3f22e9a76 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:21:58.920537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 760/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:00.004291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf02f-7a4e-427f-9544-c8700df31048 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:00.004830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf02f-7a4e-427f-9544-c8700df31048 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:00.004830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf02f-7a4e-427f-9544-c8700df31048 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:00.006095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:00.012226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c423-c60d-43d3-a450-466ad8265548 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:00.012690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c423-c60d-43d3-a450-466ad8265548 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:00.012794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c423-c60d-43d3-a450-466ad8265548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:00.032702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ca35c423-c60d-43d3-a450-466ad8265548 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:00.033924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ca35c423-c60d-43d3-a450-466ad8265548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:00.041756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:00.197567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf02f-7a4e-427f-9544-c8700df31048 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:00.212035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf02f-7a4e-427f-9544-c8700df31048 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:00.218389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:00.218983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:00.219261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:00.219535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:00.227490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:00.229102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ae0d0-1782-407e-a06f-9dfb81aa6c91 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:00.239368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 755/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:03.353665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810962-aed1-499a-b952-2e6b779f77e4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:03.354194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810962-aed1-499a-b952-2e6b779f77e4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:03.354194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810962-aed1-499a-b952-2e6b779f77e4 None None] PATH_INFO: `/v3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:03.355396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 762/1517] 10.222.0.33 () {54 vars in 925 bytes} [Wed Sep 28 07:22:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:22:03.362394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5bd-1a02-4f41-8f05-15bf016dde79 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:03.362838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5bd-1a02-4f41-8f05-15bf016dde79 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:03.362943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5bd-1a02-4f41-8f05-15bf016dde79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:03.420957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0f01f5bd-1a02-4f41-8f05-15bf016dde79 None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:03.422183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0f01f5bd-1a02-4f41-8f05-15bf016dde79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:03.439432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:22:03.492742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 756/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Wed Sep 28 07:22:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:07.936176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6ced7-9122-4088-bdd0-24c474b32467 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:07.936634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6ced7-9122-4088-bdd0-24c474b32467 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:07.936749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6ced7-9122-4088-bdd0-24c474b32467 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:07.938083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 763/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:07.942939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcecb6-0ca4-44b9-88d8-e3626a70d5eb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:07.943341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcecb6-0ca4-44b9-88d8-e3626a70d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:07.943482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcecb6-0ca4-44b9-88d8-e3626a70d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:07.964511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cfdcecb6-0ca4-44b9-88d8-e3626a70d5eb None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:07.965760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cfdcecb6-0ca4-44b9-88d8-e3626a70d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:07.974080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 757/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:22:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:22:07.982554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6ced7-9122-4088-bdd0-24c474b32467 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:07.995950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6ced7-9122-4088-bdd0-24c474b32467 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:08.002200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:08.002809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:08.003087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:08.003355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:08.012289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:08.013887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3e262-4523-4d84-a998-905c57547e5f service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:08.023639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 764/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:11.712098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32c7a-db78-4fa5-bfd4-b64caeb5f5c2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:11.712636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32c7a-db78-4fa5-bfd4-b64caeb5f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:11.712636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32c7a-db78-4fa5-bfd4-b64caeb5f5c2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:11.713992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:11.720872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb0b4-bc40-415b-a550-7c688b10a335 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:11.721377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb0b4-bc40-415b-a550-7c688b10a335 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:11.721487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb0b4-bc40-415b-a550-7c688b10a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:11.742335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-90ceb0b4-bc40-415b-a550-7c688b10a335 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:11.744408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-90ceb0b4-bc40-415b-a550-7c688b10a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:11.753601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:11.907110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb32c7a-db78-4fa5-bfd4-b64caeb5f5c2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:11.922055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb32c7a-db78-4fa5-bfd4-b64caeb5f5c2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:11.927455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:11.928042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:11.928317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:11.928587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:11.937432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:11.939006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b36e43-7655-48ff-8c00-d4313f52075d service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:11.948172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 759/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:13.013006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae22a-42a0-48d0-a22f-8bd3636f6513 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:13.013581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae22a-42a0-48d0-a22f-8bd3636f6513 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:13.013581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae22a-42a0-48d0-a22f-8bd3636f6513 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:13.014905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 07:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:13.021068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310e0e-8a28-4425-a44b-a34049d4c269 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:13.021533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310e0e-8a28-4425-a44b-a34049d4c269 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:13.021644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310e0e-8a28-4425-a44b-a34049d4c269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:13.040806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e0310e0e-8a28-4425-a44b-a34049d4c269 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:13.042029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0310e0e-8a28-4425-a44b-a34049d4c269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:13.050330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 07:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:13.201863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae22a-42a0-48d0-a22f-8bd3636f6513 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:13.215843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae22a-42a0-48d0-a22f-8bd3636f6513 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:13.221148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:13.221726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:13.221998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:13.222264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:13.230026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:13.232281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44e07c-fcd8-4177-a9a1-ee11606540f6 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:13.241337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 767/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:17.109675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:17.110262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:17.110262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:17.111521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:17.116748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeff4b-79b9-4332-9323-b4c4b45ec82d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:17.117229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeff4b-79b9-4332-9323-b4c4b45ec82d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:17.117336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeff4b-79b9-4332-9323-b4c4b45ec82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:17.137590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4fbeff4b-79b9-4332-9323-b4c4b45ec82d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:17.138874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4fbeff4b-79b9-4332-9323-b4c4b45ec82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:17.146657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:17.435895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:17.437685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:17.513805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c92924-ce13-404a-9edf-e44eceb1c64a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:17.517310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:17.517900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:17.518170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:17.518438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:17.527029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:17.528481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd7cd6-a8ec-4f19-bc66-109b32e2eb37 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:17.537605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 762/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:19.679650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca16-47b1-4cf0-9903-9efef6136c64 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:19.680490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca16-47b1-4cf0-9903-9efef6136c64 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:19.680490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca16-47b1-4cf0-9903-9efef6136c64 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:19.681852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:19.688201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc38ba-91fd-44bd-96fc-2b435f5cdf82 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:19.688618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc38ba-91fd-44bd-96fc-2b435f5cdf82 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:19.688748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc38ba-91fd-44bd-96fc-2b435f5cdf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:19.708605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ecc38ba-91fd-44bd-96fc-2b435f5cdf82 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:19.709830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ecc38ba-91fd-44bd-96fc-2b435f5cdf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:19.718391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:20.023914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eca16-47b1-4cf0-9903-9efef6136c64 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:20.035080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eca16-47b1-4cf0-9903-9efef6136c64 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:20.038504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:20.039081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:20.039353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:20.039618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:20.048189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:20.049619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293671c1-2e95-4cf7-8e5a-8cffcd682fa8 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:20.059443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 770/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:23.770328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:23.770907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:23.770907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:23.772207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:23.777355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf6e9e-9e55-46aa-a9cb-d17087c5064a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:23.777799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf6e9e-9e55-46aa-a9cb-d17087c5064a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:23.777895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf6e9e-9e55-46aa-a9cb-d17087c5064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:23.799124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-aedf6e9e-9e55-46aa-a9cb-d17087c5064a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:23.800362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aedf6e9e-9e55-46aa-a9cb-d17087c5064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:23.808473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:24.111523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:24.113233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:24.177548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:22:24.191482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8427-e2ca-4c4f-b99b-2b8dd2530f9b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:24.195048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:24.195646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:24.195920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:24.196186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:24.205767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:24.207185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362b65-c5d6-469b-b1af-8d6ceff62d94 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:24.217146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 765/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:26.336382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:26.337046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:26.337046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:26.338278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 07:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:22:26.343994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e169c-c520-4460-998b-d078a22eb262 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:26.344412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e169c-c520-4460-998b-d078a22eb262 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:26.344519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e169c-c520-4460-998b-d078a22eb262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:26.364105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3d8e169c-c520-4460-998b-d078a22eb262 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:22:26.365377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3d8e169c-c520-4460-998b-d078a22eb262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:26.373909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 07:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:22:26.677480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:22:26.679258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:22:26.694283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3986da1-2d87-466f-bd61-815ea7865642 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:22:26.699156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:22:26.700112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:22:26.700435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:22:26.701077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:22:26.711016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:22:26.712438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70baa94-f723-442f-8f55-daecf73d8637 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:22:26.721885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 773/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:22:36.359301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=11 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:22:36.717684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:52.385596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84323-3b20-447a-81a2-19f79ade168d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.386711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84323-3b20-447a-81a2-19f79ade168d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.387220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84323-3b20-447a-81a2-19f79ade168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.400305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a43bb-a4c1-4dce-8e01-6ef8c3d8bf40 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.400907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a43bb-a4c1-4dce-8e01-6ef8c3d8bf40 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.400907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a43bb-a4c1-4dce-8e01-6ef8c3d8bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.416723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2de84323-3b20-447a-81a2-19f79ade168d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.418287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2de84323-3b20-447a-81a2-19f79ade168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.431237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-430a43bb-a4c1-4dce-8e01-6ef8c3d8bf40 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.432420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-430a43bb-a4c1-4dce-8e01-6ef8c3d8bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.433673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 767/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.441685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.442282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.442776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.453756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 774/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:23:52.462497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.462783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.463058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.485226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.486785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.502940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 768/1542] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:23:52.505464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.506666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.510579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.526215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef900dc-37c2-41ff-b22a-d90cc09e1124 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.531176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 775/1543] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.533072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.538889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.539519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.539791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.540056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.540714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:52.541040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.542688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20d5ec-5989-4c7d-be31-c76586496403 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.565948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f25031-ae49-4608-a4bb-5665e222a504 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.573722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 769/1544] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:23:52.575218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.575501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.576560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.585047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.586259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.586760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.587445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.588340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:52.588871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.590190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:23:52.590190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:23:52.590190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:23:52.591689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.615986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.617400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.617732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 776/1545] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:23:52.618765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.636385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5f7a-61a5-49fc-a7e2-e3e16582fad6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.636843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 770/1546] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.640987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.652038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fa9ed-ac95-4af9-8234-081150942e09 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.653293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.654241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.654755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.655255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.656126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.658970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8b68-f801-4bd7-a30f-abb1c3be63e4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.668210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.668846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.669221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.670157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.670969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.672101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:23:52.672101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:23:52.672101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:23:52.673365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275b8f-4ad7-4eaa-8299-7cd9fb0cd115 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.750465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 777/1547] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:52.752259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.752823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.753394 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.778026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 771/1548] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/projects => generated 363 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:52.782722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e35a-5fb5-4c0a-bf25-24a0d4339366 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.783023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e35a-5fb5-4c0a-bf25-24a0d4339366 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.783301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e35a-5fb5-4c0a-bf25-24a0d4339366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.799668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.800891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.818919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cc08e35a-5fb5-4c0a-bf25-24a0d4339366 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.820307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cc08e35a-5fb5-4c0a-bf25-24a0d4339366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.881176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 778/1549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.887118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 772/1550] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.887375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.890179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.890643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.891055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.898103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fac536-d0df-4e4f-a3c7-de87fa048318 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.902709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.903307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.903584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.903891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.904639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.905672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:23:52.905672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:23:52.905672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:23:52.906925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d338ae-9c1d-4075-9a3c-a7077596d78a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.911705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.912881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.922004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 773/1551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.925964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:52.933473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 779/1552] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:52.933473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4588-5544-4651-9638-e30f625af67f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.933473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4588-5544-4651-9638-e30f625af67f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.933473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4588-5544-4651-9638-e30f625af67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.936874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e93945-7f9c-49a9-aca8-f4407765416b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:52.941492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:52.942088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.942363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.942630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.943285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:52.945189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ad1c9-813e-491a-98d8-308b27c7429a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:52.952807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-589a4588-5544-4651-9638-e30f625af67f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.954731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-589a4588-5544-4651-9638-e30f625af67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.964120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 780/1553] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:52.965416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.965763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.966143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.967922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 774/1554] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:52.969290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e6c4-1860-4603-9744-9f389d15e7a7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:52.969569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e6c4-1860-4603-9744-9f389d15e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:52.969839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e6c4-1860-4603-9744-9f389d15e7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:52.989218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.990858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.992181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d857e6c4-1860-4603-9744-9f389d15e7a7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:52.993374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d857e6c4-1860-4603-9744-9f389d15e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:52.999966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 781/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.001202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.009155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 775/1556] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.009155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.009155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.009155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.013299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786bb8-0870-48b8-8247-86452b46367f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.018211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.018843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.019117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.019386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.020057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.020358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.021991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e497fd-8a4e-48d6-adac-fe469dc6b212 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.025765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.026952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.034029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 782/1557] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.035308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.035689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.036081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.042407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 776/1558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.043235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.054035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef25cd-9751-43c3-a850-828cb92780b0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.056582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.057794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.059189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.059778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.060744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.061193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.062058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.063640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28315b14-b177-4a96-9878-413ae424e7a4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.066442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 783/1559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.067203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.073750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 777/1560] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:53.074873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53572e-ac92-479b-a1f6-9b4db9d9200e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.075162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53572e-ac92-479b-a1f6-9b4db9d9200e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.075436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53572e-ac92-479b-a1f6-9b4db9d9200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.078975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4d37-febb-4413-af64-bdaf5ca93791 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.083524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.084116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.084386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.084658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.085319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.086892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d926c-8954-4623-9bac-e9a364539b4f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.095408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ab53572e-ac92-479b-a1f6-9b4db9d9200e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.096610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ab53572e-ac92-479b-a1f6-9b4db9d9200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.097881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 784/1561] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:53.099085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.099371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.099647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.108144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 778/1562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.109379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.109658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.109927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.123598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.124811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.129033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.130220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.132792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 785/1563] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.133534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.139045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 779/1564] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.139891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.143317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d9b85-b39c-4bb7-a57a-da88e879d804 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.148012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.148605 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.148882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.149186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.149419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5a47d-8937-4872-b7b5-7ab6f6e15aa1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.149818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.151745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.154735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.155354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.155620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.155882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.156513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.156808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.158503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.171621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 780/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.179173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.182786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714442c-06c2-4709-bb5a-e63db9dbf851 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.188973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 786/1566] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.188973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.191130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.191733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.192011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.192488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.193189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.194062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.195647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.201219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.208490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 781/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.209306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.225502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2d92-3b3b-41db-9323-7fdf8a006fa3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.230041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.255458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8de34-9acc-4666-9b81-01624b85a420 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.260893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.261525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.261808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.262105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.262747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.264543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.277607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 787/1568] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:53.277607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.284143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.284750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.285057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.285320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6af4-6f01-40a7-88bf-22f4f5c24854 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.285433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.285964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.287535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.289911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.290490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.290770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.291041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.291669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.291968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.293612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.298077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 782/1569] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:53.298961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.314188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 788/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.314188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.317162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef2e5a-2b34-4afb-a83a-f2b4285af125 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.317920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684cec-a39e-4158-be1b-02e7fbb20b4e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.321735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.322340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.322596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.322724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.322890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.323188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.323463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.323582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.323726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.323830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.324394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:23:53.324693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.325419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.326570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.336877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 789/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.337699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.338738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 783/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:23:53.339531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.353851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247266-76d5-454d-8421-4319584cccc2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.354652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6cd-28c4-4ba0-bbf7-7d0e2697c6ea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.359213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.359819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.360096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.360202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.360367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.360721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.361030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.361140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.361306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.361935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.362923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.363942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.385041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 784/1573] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 284 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.385720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.393092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 790/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.393749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.398418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15610c94-e0fa-433b-8fc4-ab15e4e51696 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.455105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466a9c4-450d-4bf5-95ab-31d1904b9630 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.457015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.457744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.458085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.458361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.458992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.459776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.460457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.460771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.460964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.461112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] PATH_INFO: `/v3/projects/de797be1153940df900f635163c09d94/users/8d413510d5344e9b85b44ec40bd232a7/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.477416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=de797be1153940df900f635163c09d94, user_id=8d413510d5344e9b85b44ec40bd232a7, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.480339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.488430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 785/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.489399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.493903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 791/1576] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:53 2022] PUT /identity/v3/projects/de797be1153940df900f635163c09d94/users/8d413510d5344e9b85b44ec40bd232a7/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:53.500864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.546043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.559883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c43cef-d6f3-47ed-af06-a23623949109 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.563603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920547b-cd6a-458a-9d14-27f0a498d0ba admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.565004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.565629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.565907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.566180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] PATH_INFO: `/v3/projects/3be8275d006449a0b8e9f59f1a7cbaeb/users/4c935dbf55374d18a2d9ffde6b1fa099/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.568265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.568492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.568856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.569158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.569495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.570189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.572174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.584421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be8275d006449a0b8e9f59f1a7cbaeb, user_id=4c935dbf55374d18a2d9ffde6b1fa099, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.585519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:23:53.585519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:23:53.585519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:23:53.587780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.602821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 792/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.602821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.603087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 786/1578] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:53 2022] PUT /identity/v3/projects/3be8275d006449a0b8e9f59f1a7cbaeb/users/4c935dbf55374d18a2d9ffde6b1fa099/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:53.603087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.669736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0be9b-1655-4417-82e7-e44673fe0ba6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.669956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb498f-0c80-4325-aedd-335578ca4d74 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.674128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.674431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.674726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.674999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.675116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.675255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.675373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.675545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.675877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.676171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.677827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.678114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.712025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 787/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.712874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.719457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 793/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.719457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.776267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac11105-f5f7-4571-9cf8-7332c7832562 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.781631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.781934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18327e6a-245d-4813-a960-6a39bff5d11d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.782227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.782500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.782767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.783398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.785047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.786660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.787259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.787538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.787816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.788456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.790439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4fdd2-47b3-4d22-9181-0810f1853b5e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.795939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 788/1581] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:53.796786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.809046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368658d-76f1-47ee-8d7e-5996de42001d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.816364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 794/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.816364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.816612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.816750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.816750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.816914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.816914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.816914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.816914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.818602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.839763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 789/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.840570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.847406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] MFA Rules not processed for user `8d413510d5344e9b85b44ec40bd232a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.848991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.851210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b258a70-686a-49d8-b6b4-b995fb6a8840 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.858819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.859423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.859729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.859997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.860678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.862597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea490925-0160-4b1d-980e-d91e94b5d1a7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.885132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.888547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 790/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.888547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.888547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.888547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.915459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] MFA Rules not processed for user `4c935dbf55374d18a2d9ffde6b1fa099`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:53.916649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:53.920073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 795/1585] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.921658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.932069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257b153-744a-4771-835c-be12a7eb7a77 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.934380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:53.937663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.938237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.938547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.938798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.939430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.941490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.962571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 796/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:53.963220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:53.973170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9ccb-3c33-467e-8068-f49134bab566 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:53.977751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:53.978337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:53.978616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:53.978886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:53.979561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:53.981537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:53.992747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 791/1587] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:53.993770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.005384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5f699-a1c6-41f7-9c0d-6d675fde3594 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.009664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 797/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:53 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:54.009664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.013501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.014175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.014447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.014712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.015352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.016929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.021276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d35c-44f6-43d8-96dd-d87ff8879140 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.026529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.030603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 792/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.030752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.030752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.030919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.031004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.031004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.031004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.040475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b78d-c0fd-4c61-a400-bebe7a018898 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.044345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 798/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.044345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.048225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.048814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.049123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.049387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.050010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.051558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.052522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c687a-4109-45e3-ab82-ceef3958c78b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.057495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.058071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.058345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.058611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.059245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.060998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.062914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 793/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.063683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.071821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 799/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.072530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.074042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47972c73-0ab0-4a99-ae27-02d645f3c952 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.079266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.079856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.080129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.080402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.081059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.082927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.084391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b98f1-c929-48e9-ae3b-7d1b15dbf11f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.088920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.089662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.089941 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.090207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.090861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.093255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.106672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 800/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.106672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.114698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f355e4-97d7-469f-9054-a486190b9ac0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.118416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 794/1594] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:54.118416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.174215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.175248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.175529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.175817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.178189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d76e-8aff-4265-a8dc-94baf6566abb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.182506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.183047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.183647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.183919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.184186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.184820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.184997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.186619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.198216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 795/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.199028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.210617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a11700-d90f-4e0a-a1b9-864702db24f1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.215845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 801/1596] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:54.215845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.218963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.219687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.219986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.220248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] PATH_INFO: `/v3/projects/776c9f40cffa41ddbc8d35a95627504a/users/41137e97886749c79dac68052f8b0ef0/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.235404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=776c9f40cffa41ddbc8d35a95627504a, user_id=41137e97886749c79dac68052f8b0ef0, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.243693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.257970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 796/1597] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/776c9f40cffa41ddbc8d35a95627504a/users/41137e97886749c79dac68052f8b0ef0/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.258860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.271969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.288615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3d5a7-18c6-4414-a576-7eae664ca2c7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.308174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.323445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6044-9cc4-4d41-b8e9-e45b7d1bb6c5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.328248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.328853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.329161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.329435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.330076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.331661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.342033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 797/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.342899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.348965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.349565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.349838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.350108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.350739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.352325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.354961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48d103-9913-42af-aaec-43b77979c76c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.361069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.361658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.361962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.362231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.362841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 802/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.362955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.363701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.364445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.374127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda033b-706a-46f9-b366-0dd314d11856 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.378372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 798/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.378372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.379280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.379890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.380171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.380471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.381174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.382802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.385806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7442c1a-e6bb-4d77-b620-2731d9a13a0f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.390519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.391094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.391370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.391640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.392257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.396209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 803/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.396209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.396454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.403032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b6f5f-4ac3-45d1-a92a-6b86762411d5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.408235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.408867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.409190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.409458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.410085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.411885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1ed71-f4d0-4c24-a910-161cbebbf8e6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.415975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 799/1602] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:54.416869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.425838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 804/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.425838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.425838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.425838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.428439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507220ec-22d4-4827-833a-b478fbe1a4eb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.433290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.433879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.434156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.434426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.435749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.438098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.452568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] MFA Rules not processed for user `41137e97886749c79dac68052f8b0ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:54.453949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:54.459987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 800/1604] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:23:54.460780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.471620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d2bf-59d1-4cab-b9df-3127fb187866 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.477159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.477871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.478171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.478482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] PATH_INFO: `/v3/projects/6f391a1187074b68b3c6a887634b479b/users/269ef35368734dcd9d3cc901db39fe10/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.495481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f391a1187074b68b3c6a887634b479b, user_id=269ef35368734dcd9d3cc901db39fe10, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.499130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.511932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 801/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/6f391a1187074b68b3c6a887634b479b/users/269ef35368734dcd9d3cc901db39fe10/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.512706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.524698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 805/1606] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:54.525571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.536904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89f284-0e72-4eaa-a175-8619d2a57782 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.577293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e6e5-a8e0-409d-970e-76588a0c7e05 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.582623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.583305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.583609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.583941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] PATH_INFO: `/v3/projects/fc7675d275394d8893d0facf20d3a4e7/users/d286b1e0afe9413291e0e2503b4c204a/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.597189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.597848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.598125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.598397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] PATH_INFO: `/v3/projects/416f25107a0d4eb0bab9d92e59cf07f4/users/c0983c1e9f7649c2890000959cea452b/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.600832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7675d275394d8893d0facf20d3a4e7, user_id=d286b1e0afe9413291e0e2503b4c204a, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.601851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.603923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.615354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=416f25107a0d4eb0bab9d92e59cf07f4, user_id=c0983c1e9f7649c2890000959cea452b, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.618053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.618288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 802/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/fc7675d275394d8893d0facf20d3a4e7/users/d286b1e0afe9413291e0e2503b4c204a/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.619070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.635337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 806/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/416f25107a0d4eb0bab9d92e59cf07f4/users/c0983c1e9f7649c2890000959cea452b/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.637813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.676486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.683497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7458e6e-2149-42a9-a4b2-e62d996495d9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.703171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9779-5e4b-4aef-8964-25c729f18f98 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.709163 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.709769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.710185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.710459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.711222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.712835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.723177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 807/1609] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.724045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.734107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fc36-6d51-475b-9974-bb62d83495a1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.738807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.739438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.739717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.739989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] PATH_INFO: `/v3/projects/60646ef7a4ad4eb1860952504092742d/users/764cfe23613e4a97ba05f8bf0ed77ef1/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.744975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.745637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.745913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.746180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] PATH_INFO: `/v3/projects/fb4b67535ab1418b9475c9199f374c92/users/1f7adc6bcf494189a77341fb643f2fda/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.754476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60646ef7a4ad4eb1860952504092742d, user_id=764cfe23613e4a97ba05f8bf0ed77ef1, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.757141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.760656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4b67535ab1418b9475c9199f374c92, user_id=1f7adc6bcf494189a77341fb643f2fda, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.763927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f6c27-5e75-4f26-9b97-76b640de5dbc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.775661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 808/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/60646ef7a4ad4eb1860952504092742d/users/764cfe23613e4a97ba05f8bf0ed77ef1/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.776532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.776881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 803/1611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/fb4b67535ab1418b9475c9199f374c92/users/1f7adc6bcf494189a77341fb643f2fda/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:54.778193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.778586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.779396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.800704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:54.802239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:54.841709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7913e30-cd6a-490e-9a6e-140f6a073405 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.862109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 804/1612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:54.862949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.899313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.899932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.900190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.900459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.901125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.902711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410f3c2-19e1-48f5-ad5a-ccead35231c3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:54.908493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.913922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 809/1613] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:54.915196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf52f-733e-429b-9835-92f9181f77a9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.915489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf52f-733e-429b-9835-92f9181f77a9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.916288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf52f-733e-429b-9835-92f9181f77a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.921581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745489f3-f804-4e41-9053-5f239547e1a5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.925188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.925797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.926073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.926343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.935908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-03ecf52f-733e-429b-9835-92f9181f77a9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:54.937078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-03ecf52f-733e-429b-9835-92f9181f77a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:54.944519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 810/1614] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:54.945791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.946073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.946338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.965061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:54.966330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:54.970656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:54.974925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 811/1615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:54.975791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:54.987546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ce750-5da5-460a-bbb2-e3807835a226 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:54.992430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:54.992587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:54.993117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:54.993395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:54.993662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] PATH_INFO: `/v3/projects/87e693b4ca95412db65788d16efc04c8/users/2a789ef1ae0d47c3bf85da2a668d7a8a/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:54.994040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a8ebe-846b-40d5-8352-83f44d44dfc3 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.003081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 805/1616] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.004384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.004672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.004966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.010216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e693b4ca95412db65788d16efc04c8, user_id=2a789ef1ae0d47c3bf85da2a668d7a8a, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.013611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2de7c-62b6-4d33-8575-6e950ce615d3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.025399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:55.026611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:55.030443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 812/1617] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:54 2022] PUT /identity/v3/projects/87e693b4ca95412db65788d16efc04c8/users/2a789ef1ae0d47c3bf85da2a668d7a8a/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:55.031389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.031663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.031937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.060864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] MFA Rules not processed for user `1f7adc6bcf494189a77341fb643f2fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:55.062140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:55.108632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 806/1618] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:55.109852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.145742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 813/1619] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:55.146634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.160279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef8c1-d0f2-42b0-ad2b-b3032430df4c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.167115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.168653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.168973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.169253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] PATH_INFO: `/v3/projects/e244df106ee04da7960526dd44780549/users/4354eb5589a74568924c0f0e63128249/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.177845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753959cb-88a7-49b4-a26f-9072378e19c4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.183156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.183755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.184097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.184533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.193627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e244df106ee04da7960526dd44780549, user_id=4354eb5589a74568924c0f0e63128249, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.196328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.197747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.197889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.210714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 807/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.211548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.217127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 814/1621] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:55 2022] PUT /identity/v3/projects/e244df106ee04da7960526dd44780549/users/4354eb5589a74568924c0f0e63128249/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:55.217127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.227080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b0dd4-d76b-409d-b624-5e9c12f365be service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.284170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dbbb-00be-47fd-a0a6-db0251f6f509 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.285179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.286033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.286570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.287059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.294221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.295972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.296388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.297245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.340523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:55.349674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:55.378615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.378816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.380342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.381065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.393015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 815/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.393860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.395356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 808/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.396305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.403289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720498d4-3e7e-489f-b536-1e9ddbce904e service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.407797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f407b4-0cf8-4c72-9d53-fabe05c30c52 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.409332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.409986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.410276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.410545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.411187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.412842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ff264-b9e3-4380-bbaa-47d71600174c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.413419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.414295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.414882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.415584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.427705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 816/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:55.427705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad50c4-a59b-4219-b88b-7d560cf95554 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.427705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad50c4-a59b-4219-b88b-7d560cf95554 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.427705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad50c4-a59b-4219-b88b-7d560cf95554 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.428105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.428195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 817/1625] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:23:55.428976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad50c4-a59b-4219-b88b-7d560cf95554 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.429569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.442689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad50c4-a59b-4219-b88b-7d560cf95554 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.444726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 809/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.445838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.449240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.450012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.450387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.450795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.451567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.453481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90b62d-e559-4925-a000-326a023a922b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.457294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae42180-2e2b-4527-bfe9-48f4a3ed60cf service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.465791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 818/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:23:55.467818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1092c-67aa-4405-93bb-aead08e472ca None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.468109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1092c-67aa-4405-93bb-aead08e472ca None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.468391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1092c-67aa-4405-93bb-aead08e472ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.486255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.487524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.488535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.489127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] PATH_INFO: `/v3/projects/87e693b4ca95412db65788d16efc04c8/users/2a789ef1ae0d47c3bf85da2a668d7a8a/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.492085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e693b4ca95412db65788d16efc04c8, user_id=2a789ef1ae0d47c3bf85da2a668d7a8a, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.496176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.512903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 810/1628] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:55 2022] PUT /identity/v3/projects/87e693b4ca95412db65788d16efc04c8/users/2a789ef1ae0d47c3bf85da2a668d7a8a/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:55.522058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.522312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-88b1092c-67aa-4405-93bb-aead08e472ca None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:55.522312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-88b1092c-67aa-4405-93bb-aead08e472ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:55.595500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8aa11-91c4-407a-8489-c06df51d851d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.603114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.603923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.604199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.605349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] PATH_INFO: `/v3/projects/e244df106ee04da7960526dd44780549/users/4354eb5589a74568924c0f0e63128249/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.610230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e244df106ee04da7960526dd44780549, user_id=4354eb5589a74568924c0f0e63128249, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.613487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac9a6b-c811-40a3-a28e-93728a871342 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.627946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 811/1629] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:23:55 2022] PUT /identity/v3/projects/e244df106ee04da7960526dd44780549/users/4354eb5589a74568924c0f0e63128249/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:55.629717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.630051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.630386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.634042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 819/1630] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:23:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:55.651045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.651421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.651700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.686454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] MFA Rules not processed for user `2a789ef1ae0d47c3bf85da2a668d7a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:55.687655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:55.699949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] MFA Rules not processed for user `4354eb5589a74568924c0f0e63128249`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:55.699949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:55.797261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 812/1631] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:55.798707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.802962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 820/1632] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:55.814353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.834350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:55.845836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:55.894256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47bf06-f4ae-486b-8f48-52a556c8b240 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.901831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.902565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.902965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.903256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.905863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae82b0-220b-4846-a8b0-ac9d10b9500b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.911774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.912716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.912856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.913114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.927336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.929576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.944918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 821/1633] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:23:55.945669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:55.959221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2310ff-4608-420c-84be-ef85522cc7ed service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:55.969173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:55.971419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:55.972044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:55.972664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:55.973206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.975128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:55.987332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:55.987672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 813/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:55.989361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1755b15-284c-4e46-99be-458fb34d1365 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:56.001570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 822/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:56.119318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:56.137799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99b2b-6d90-4ecd-9880-72b0c0b05afa service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:56.150139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:56.151435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.151719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.153019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.295994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:56.302980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809850ba-3fd9-4e7b-a4f7-77f91a185cd0 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:56.329455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 814/1636] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:56.847304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc35db2-511a-4e1c-ac46-0e80a26402ca None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.847822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc35db2-511a-4e1c-ac46-0e80a26402ca None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.847822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc35db2-511a-4e1c-ac46-0e80a26402ca None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.849505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 823/1637] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:23:56.859913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8868-b86d-4956-ab48-7777b2437946 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.860473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8868-b86d-4956-ab48-7777b2437946 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.860473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8868-b86d-4956-ab48-7777b2437946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.890301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9b7c8868-b86d-4956-ab48-7777b2437946 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:56.891644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9b7c8868-b86d-4956-ab48-7777b2437946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:56.901949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc35db2-511a-4e1c-ac46-0e80a26402ca None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:56.931024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 815/1638] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:23:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:56.932555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d729-0640-460c-9de0-c462c19bacb0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.933023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d729-0640-460c-9de0-c462c19bacb0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.933438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d729-0640-460c-9de0-c462c19bacb0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.934897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 816/1639] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:23:56.936902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f99dc-9729-4619-b5fc-934aba31ce8c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.937390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f99dc-9729-4619-b5fc-934aba31ce8c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.937805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f99dc-9729-4619-b5fc-934aba31ce8c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.939317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 817/1640] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:23:56.941342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb81a7-1f79-4579-9a57-aa5b423df2c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.953444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb81a7-1f79-4579-9a57-aa5b423df2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.954003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb81a7-1f79-4579-9a57-aa5b423df2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:56.975210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e9cb81a7-1f79-4579-9a57-aa5b423df2c9 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:56.977660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e9cb81a7-1f79-4579-9a57-aa5b423df2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:56.989496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 818/1641] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:23:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:56.991202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:56.991645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:56.992151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.017075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:57.018791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:57.031137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 819/1642] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:23:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:57.032247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:57.053681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3393f-03d1-454b-8baa-8c06ec28d617 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:57.055679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc35db2-511a-4e1c-ac46-0e80a26402ca None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:57.058315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:57.058931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.059216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.059506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.064601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:57.068555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.068938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.069340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] PATH_INFO: `/v3/users/4c935dbf55374d18a2d9ffde6b1fa099` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.075174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c935dbf55374d18a2d9ffde6b1fa099)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:57.080519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:57.080872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:57.082438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:57.097860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 820/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:57.099515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:57.113086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e9fd-79a8-4670-bb7b-b325cce2bb5d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:57.116393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:57.117280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.117655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.118122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.132063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:57.133758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e78fd1-5d54-470a-97a6-e3bfe1376119 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:57.150795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 821/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:57.201948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c935dbf55374d18a2d9ffde6b1fa099'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.201948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c935dbf55374d18a2d9ffde6b1fa099'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.218731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c935dbf55374d18a2d9ffde6b1fa099'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.218731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c935dbf55374d18a2d9ffde6b1fa099'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.260992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c935dbf55374d18a2d9ffde6b1fa099'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.304445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 824/1645] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:23:56 2022] DELETE /identity/v3/users/4c935dbf55374d18a2d9ffde6b1fa099 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:23:57.310452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.310836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.311245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.426760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:57.430670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:57.465805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:57.467400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:57.468922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:57.537443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:57.540783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 822/1646] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:23:57.566796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:57.572211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:57.609271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b114-c7c3-43a3-9c37-5835763f67e2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:57.623804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:57.624633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.624999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.625468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] PATH_INFO: `/v3/users/1f7adc6bcf494189a77341fb643f2fda` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.627338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7adc6bcf494189a77341fb643f2fda)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:57.629433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:57.739319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf803fa5-6c00-4f46-9d00-fa7348323ad3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:57.743702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7adc6bcf494189a77341fb643f2fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.744277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7adc6bcf494189a77341fb643f2fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.753175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:57.753175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.753175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.753935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.762007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7adc6bcf494189a77341fb643f2fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.762742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7adc6bcf494189a77341fb643f2fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.782165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:57.783953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c980492-6d4f-48b6-807a-4f59da620c3a service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:57.791322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7adc6bcf494189a77341fb643f2fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:57.814039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 823/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:23:57.824231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 825/1648] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:23:57 2022] DELETE /identity/v3/users/1f7adc6bcf494189a77341fb643f2fda => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:57.825980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:57.826426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:57.826815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:57.915930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:57.915930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:58.051945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 824/1649] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:23:58.112982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:58.114756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:58.265810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc2e35-b806-4b60-a12d-adaa58e4ace5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:58.273576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:58.274223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:58.274501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:58.274777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:58.296367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:58.299410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc221686-951e-40b7-a724-665418d1fe96 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:58.325402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 826/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:23:59.491269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:59.494090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:59.540930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4610d3-7c2c-4048-a649-4c7280087227 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:23:59.555397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:23:59.556065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:59.557360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:59.559490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] PATH_INFO: `/v3/projects/fb4b67535ab1418b9475c9199f374c92` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:59.562929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4b67535ab1418b9475c9199f374c92)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:23:59.570487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:23:59.720532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4b67535ab1418b9475c9199f374c92', 'request_id': 'req-843d9272-1ee9-41d6-814b-29917444a466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:59.735606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:59.740911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4b67535ab1418b9475c9199f374c92', 'request_id': 'req-843d9272-1ee9-41d6-814b-29917444a466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:59.755217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4b67535ab1418b9475c9199f374c92', 'request_id': 'req-843d9272-1ee9-41d6-814b-29917444a466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:59.784014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] The token cache is being invalidate because project fb4b67535ab1418b9475c9199f374c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:23:59.784756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:59.786272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:23:59.788773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 825/1651] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:23:59 2022] DELETE /identity/v3/projects/fb4b67535ab1418b9475c9199f374c92 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:23:59.794960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fb21-c008-4f1c-b55f-67acc67a2c01 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:23:59.795552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fb21-c008-4f1c-b55f-67acc67a2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:23:59.795968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fb21-c008-4f1c-b55f-67acc67a2c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:23:59.842945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-34e6fb21-c008-4f1c-b55f-67acc67a2c01 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:23:59.845280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34e6fb21-c008-4f1c-b55f-67acc67a2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:59.938542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:23:59.953682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:23:59.956741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:23:59.990275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 827/1652] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:00.004310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.004774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.005179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.050292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:00.052796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:00.071413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 828/1653] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:00.077270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.094997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88033edb-c1c9-4670-91ab-c7c30f731614 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.109080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.111072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.111635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.112049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.113047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:00.114656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.119908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.138426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d9272-1ee9-41d6-814b-29917444a466 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.152122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 829/1654] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:00.152929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.153964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.154437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.155486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] PATH_INFO: `/v3/projects/3be8275d006449a0b8e9f59f1a7cbaeb` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.156929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.161449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be8275d006449a0b8e9f59f1a7cbaeb)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.166752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.181219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d1cf1-4acd-4257-a9fb-a0d0bb175eca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.196795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.197634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.198136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.198536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.199584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.204849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.273342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 830/1655] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:00 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:00.278168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.301747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be8275d006449a0b8e9f59f1a7cbaeb', 'request_id': 'req-e71d175d-11ae-4628-9f28-396e03f684c9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.326965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be8275d006449a0b8e9f59f1a7cbaeb', 'request_id': 'req-e71d175d-11ae-4628-9f28-396e03f684c9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.348465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be8275d006449a0b8e9f59f1a7cbaeb', 'request_id': 'req-e71d175d-11ae-4628-9f28-396e03f684c9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.372058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] The token cache is being invalidate because project 3be8275d006449a0b8e9f59f1a7cbaeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:00.373069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.375328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.378264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 826/1656] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:23:59 2022] DELETE /identity/v3/projects/3be8275d006449a0b8e9f59f1a7cbaeb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:00.382166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.385708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:00.451679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc714b-45ae-428a-bb0f-4d1fbcc6533a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.463534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.464161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.464437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.464703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.465379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.467547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.546656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 831/1657] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:00 2022] POST /identity/v3/users => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:00.549247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.551349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:00.551489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d175d-11ae-4628-9f28-396e03f684c9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.566128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.566869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.567153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.567431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] PATH_INFO: `/v3/users/4354eb5589a74568924c0f0e63128249` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.570351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4354eb5589a74568924c0f0e63128249)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.574183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:24:00.574183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Sep 28 07:24:00.574183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: Sep 28 07:24:00.578075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.594183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6b6d-fd40-4bb5-8f4a-68ac6f830706 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.611678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.611678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.611678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.619011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.622780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.625622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.649248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:00.684415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 832/1658] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:00.694961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.766385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f9ba2-18aa-4171-8b5d-6bd5f866edaa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.798168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.799622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.800172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.800551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.801387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.803686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:00.823438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4354eb5589a74568924c0f0e63128249'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.826979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4354eb5589a74568924c0f0e63128249'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.828270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4354eb5589a74568924c0f0e63128249'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.843383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 833/1659] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:00.844487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:00.849903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:00.852964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4354eb5589a74568924c0f0e63128249'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.883152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cac999-b6b4-4571-ae70-957a5d39fd03 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:00.884746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4fcc645e-e751-45da-ad3e-326c6478c318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4354eb5589a74568924c0f0e63128249'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:00.890796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:00.895631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:00.896032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:00.896486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] PATH_INFO: `/v3/projects/65a56e3dd3fe4554a215b003fe095afa/users/55fe561e4b644243848b1bcc10e34e18/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:00.924794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 827/1660] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:00 2022] DELETE /identity/v3/users/4354eb5589a74568924c0f0e63128249 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:00.958656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a56e3dd3fe4554a215b003fe095afa, user_id=55fe561e4b644243848b1bcc10e34e18, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:00.965944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:01.059066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 834/1661] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:00 2022] PUT /identity/v3/projects/65a56e3dd3fe4554a215b003fe095afa/users/55fe561e4b644243848b1bcc10e34e18/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:01.073697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:01.074241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:01.074707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:01.166842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] MFA Rules not processed for user `55fe561e4b644243848b1bcc10e34e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:01.170482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:01.338669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 828/1662] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:01.343903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:01.504642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:01.575288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:01.577390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:01.590248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a6-4aa6-4224-82f9-0ddf205f0cec admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:01.600717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:01.601530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:01.601956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:01.602519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:01.603317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:01.605918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:01.619393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd15f1-bbd9-4d2b-a670-e9d3cd65342c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:01.625599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:01.626276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:01.626555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:01.626827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] PATH_INFO: `/v3/projects/e244df106ee04da7960526dd44780549` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:01.628161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e244df106ee04da7960526dd44780549)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:01.648421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:01.667611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 835/1663] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:01 2022] POST /identity/v3/projects => generated 405 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:01.669782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:01.831464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e244df106ee04da7960526dd44780549', 'request_id': 'req-04e59a54-fed7-4941-9cdf-5e5a65bc471e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:01.853538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e244df106ee04da7960526dd44780549', 'request_id': 'req-04e59a54-fed7-4941-9cdf-5e5a65bc471e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:01.878187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e244df106ee04da7960526dd44780549', 'request_id': 'req-04e59a54-fed7-4941-9cdf-5e5a65bc471e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:01.897746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ea176-41f6-4867-94ff-741630388e13 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:01.904048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] The token cache is being invalidate because project e244df106ee04da7960526dd44780549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:01.906723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:01.907659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-04e59a54-fed7-4941-9cdf-5e5a65bc471e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:01.911277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 829/1664] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:01 2022] DELETE /identity/v3/projects/e244df106ee04da7960526dd44780549 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:01.917125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:01.917603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:01.918016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:01.941687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:01.961661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:01.966223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:02.096611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.097494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.097934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.098404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.099367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.102597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7ef72-f66f-4b69-83b1-c76a18a90eaa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.176551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 830/1665] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:02.178532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 836/1666] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:01 2022] POST /identity/v3/users => generated 309 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:02.179511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.182013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.182383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.182559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.187154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:02.222512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4282076-76f3-431d-846a-d64cfbf1daed None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.235759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.236574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.236857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.237153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.237827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.242488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.244123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:02.246057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:02.273569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 837/1667] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:02.274543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.278910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 831/1668] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:02.281912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.317815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760f56a-129c-496f-9a5d-4e5c342052f4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.320571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe76da-1f91-4d6e-9177-35d465b2b1c5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.329405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.331237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.331894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.332168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.332439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] PATH_INFO: `/v3/projects/a4b8ea15e151462e98a707cead3768a1/users/6c7d581f82b84594ad652728fbdfe259/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.333110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.333428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.333689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.334385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:02.334686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.339166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.369880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 838/1669] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:02.370695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.375551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b8ea15e151462e98a707cead3768a1, user_id=6c7d581f82b84594ad652728fbdfe259, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.378306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.395651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415c0f3-1c38-4827-94aa-1167d54ef6a4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.401628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.415836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 832/1670] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:02 2022] PUT /identity/v3/projects/a4b8ea15e151462e98a707cead3768a1/users/6c7d581f82b84594ad652728fbdfe259/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:02.416043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.416043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.416043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.416043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.417660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.422122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.503515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 839/1671] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:02 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:02.517581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.632116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da2ef-b052-45fb-8401-6cd757a6f384 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.641016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:02.728442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a6b6-1e23-4c71-aa50-49d17d7ace4d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.744814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.745664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.745938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.746217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.746861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.753243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.809612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 840/1672] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:02 2022] POST /identity/v3/users => generated 294 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:02.813297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.836557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3da5-898a-443e-a5af-7ba612c698e1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.840109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.840729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.841367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.841547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.842247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.842351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.842634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.842938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.843047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.843918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.844553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.845820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.877160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 841/1673] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:02.878169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.890111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 833/1674] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:02.891097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.900356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182329-88b1-4df5-907a-7e036e5b7c77 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.907152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.908113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.908675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.909181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.910204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.911284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d41486-983b-4e1b-bce2-122f9b089d35 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.912416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.929798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:02.932677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 842/1675] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:02.932886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.933098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.933383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] PATH_INFO: `/v3/projects/a4b8ea15e151462e98a707cead3768a1/users/6c7d581f82b84594ad652728fbdfe259/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:02.933854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:02.938751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b8ea15e151462e98a707cead3768a1, user_id=6c7d581f82b84594ad652728fbdfe259, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:02.949851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419f25d-3556-47a0-a1dd-53735d9326fe admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:02.960511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2aecc-b737-422a-8c32-5f27f3d6fe24 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:02.965402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:02.982335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 834/1676] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:02 2022] PUT /identity/v3/projects/a4b8ea15e151462e98a707cead3768a1/users/6c7d581f82b84594ad652728fbdfe259/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:02.994349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:02.994656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:02.994903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.060846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] MFA Rules not processed for user `6c7d581f82b84594ad652728fbdfe259`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:03.062211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:03.101921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:03.105489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:03.105788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:03.107045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] PATH_INFO: `/v3/projects/4dac84ced24a4030991c3e7bc5854227/users/d470a038cd5245ec900b2836e701d350/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.137474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dac84ced24a4030991c3e7bc5854227, user_id=d470a038cd5245ec900b2836e701d350, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:03.141827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433de5f3-79a4-47a7-95cf-95f5041c6e18 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:03.175774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 843/1677] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:02 2022] PUT /identity/v3/projects/4dac84ced24a4030991c3e7bc5854227/users/d470a038cd5245ec900b2836e701d350/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:03.187427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:03.187745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:03.188219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.192547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 835/1678] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:03.205010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:03.209301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:03.267548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] MFA Rules not processed for user `d470a038cd5245ec900b2836e701d350`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:03.268797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:03.348498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0ac71-2df4-4199-8047-7490ab4daaa0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:03.414983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 844/1679] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:03.415953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:03.465057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:03.492249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:03.492865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:03.493167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:03.493445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.571096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93567e0-9f9a-4a00-b063-83590905257a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:03.578037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:03.578946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:03.578946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:03.579197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.579994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:03.583356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:03.592887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:03.613329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:03.614226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:03.634143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 845/1680] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:03 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:03.638669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:03.640899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 836/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:03 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:03.881767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370131d1-4fdf-4340-b986-6ff7dc05a2f9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:03.891260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:03.891902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:03.895258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:03.895572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:03.897871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:03.903081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:03.975124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 846/1682] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:03 2022] POST /identity/v3/users => generated 300 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:03.980266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.000714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03e94c-bb04-4218-81d5-b3983e5ecebc service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.016390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.018887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.022855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.026701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.029085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.031256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.037516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bc26f-da84-40cf-a1d6-aa9b4168da84 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.056811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.057545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.058254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.058588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] PATH_INFO: `/v3/users/55fe561e4b644243848b1bcc10e34e18` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.061406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fe561e4b644243848b1bcc10e34e18)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.063853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:04.064823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:04.135877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 837/1683] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:04.141477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.189227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fe561e4b644243848b1bcc10e34e18'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:04.191537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fe561e4b644243848b1bcc10e34e18'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:04.192491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289b134-dffe-43a0-a684-ce419bc475e0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.211529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fe561e4b644243848b1bcc10e34e18'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:04.211903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.212360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fe561e4b644243848b1bcc10e34e18'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:04.212584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.212867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.213171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] PATH_INFO: `/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec/users/34991d45201c43938b347a4e8c91502c/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.248336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fe561e4b644243848b1bcc10e34e18'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:04.254298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d0e074e9c845aca8a38bfcd902f6ec, user_id=34991d45201c43938b347a4e8c91502c, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.259935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88509-9aaa-4256-b44a-922111987804 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:04.271709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:04.282874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 847/1684] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:04 2022] DELETE /identity/v3/users/55fe561e4b644243848b1bcc10e34e18 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:04.290296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 838/1685] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:04 2022] PUT /identity/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec/users/34991d45201c43938b347a4e8c91502c/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:04.291613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.293319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:04.293869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.294587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.295339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.322181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:04.323651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:04.465164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4521f8-3549-4b39-8c90-fe911ff5778b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.473180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.476461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.477224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.477559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.479779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.486494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:04.503385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 839/1686] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:04.507438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.510243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:04.519491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 848/1687] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:04.522929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.553748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80ad50-2f94-4fd7-993e-442abe8bbb0b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.566799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.570750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.571150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.571383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] PATH_INFO: `/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec/users/34991d45201c43938b347a4e8c91502c/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.577322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d0e074e9c845aca8a38bfcd902f6ec, user_id=34991d45201c43938b347a4e8c91502c, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.583796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea4274-7054-48cc-9949-de34c74de131 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:04.608741 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 849/1688] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:04 2022] PUT /identity/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec/users/34991d45201c43938b347a4e8c91502c/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:04.619403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.619622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.621549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.661174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836062c-a7ea-46e6-885f-a065da4bf61e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:04.692398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] MFA Rules not processed for user `34991d45201c43938b347a4e8c91502c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:04.696207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:04.819755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:04.820396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:04.820676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:04.820976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:04.845349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 850/1689] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:04.859043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:04.863175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:04.901658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:04.991261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:04.994848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:05.004705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce8535-4fb5-419f-91af-6905fd0f0b38 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:05.015512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:05.018239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:05.018539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:05.018814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:05.024662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 840/1690] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:05.034227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:05.035785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369ba76-9a52-4049-ac3c-06383742af0f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:05.049253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 851/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:05.156559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:05.179574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96e73-3245-47cf-958e-1d65fa7beb6d service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:05.195347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:05.196023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:05.196302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:05.196578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] PATH_INFO: `/v3/users/d470a038cd5245ec900b2836e701d350` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:05.205184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d470a038cd5245ec900b2836e701d350)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:05.208249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:05.378015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd470a038cd5245ec900b2836e701d350'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.378015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd470a038cd5245ec900b2836e701d350'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.380565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd470a038cd5245ec900b2836e701d350'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.410925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd470a038cd5245ec900b2836e701d350'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.485147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd470a038cd5245ec900b2836e701d350'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.522632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 841/1692] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:05 2022] DELETE /identity/v3/users/d470a038cd5245ec900b2836e701d350 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:05.529698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:05.530201 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:05.530661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:05.582509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:05.584450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:05.597611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:05.601124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:05.618362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:05.790222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 852/1693] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:05.797297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289190b-10b4-4100-83de-cf34198a6f46 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:05.800336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:05.801562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:05.819074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:05.820592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:05.820592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:05.820592 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] PATH_INFO: `/v3/projects/65a56e3dd3fe4554a215b003fe095afa` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:05.829104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a56e3dd3fe4554a215b003fe095afa)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:05.834911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:05.954629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:05.973114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a56e3dd3fe4554a215b003fe095afa', 'request_id': 'req-954e5b08-37c4-44ea-8d5b-873ce5f85194'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:05.980282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a56e3dd3fe4554a215b003fe095afa', 'request_id': 'req-954e5b08-37c4-44ea-8d5b-873ce5f85194'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.003082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a56e3dd3fe4554a215b003fe095afa', 'request_id': 'req-954e5b08-37c4-44ea-8d5b-873ce5f85194'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.032245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] The token cache is being invalidate because project 65a56e3dd3fe4554a215b003fe095afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:06.032686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.033489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.036192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 842/1694] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:05 2022] DELETE /identity/v3/projects/65a56e3dd3fe4554a215b003fe095afa => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:06.040168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:06.046213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:06.046943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9325f-3ba0-4d9e-ae22-a72e3cdaa885 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:06.164042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:06.164671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:06.164971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:06.168378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:06.178258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:06.226901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5b08-37c4-44ea-8d5b-873ce5f85194 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:06.241047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:06.241733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:06.242080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:06.242434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] PATH_INFO: `/v3/users/6c7d581f82b84594ad652728fbdfe259` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:06.243682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c7d581f82b84594ad652728fbdfe259)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:06.250978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:06.367794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:06.370159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:06.370747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c7d581f82b84594ad652728fbdfe259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.373605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c7d581f82b84594ad652728fbdfe259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.375765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c7d581f82b84594ad652728fbdfe259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.389985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 853/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:06.395464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c7d581f82b84594ad652728fbdfe259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.428236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c7d581f82b84594ad652728fbdfe259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:06.448984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 843/1696] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:06 2022] DELETE /identity/v3/users/6c7d581f82b84594ad652728fbdfe259 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:06.912649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:06.916843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:06.943024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e50d-a91e-496e-b2b9-1ae2d9a037fe service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:06.972680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:06.979919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:06.981519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.051806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:07.085976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.086720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.087350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.087739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] PATH_INFO: `/v3/users/41137e97886749c79dac68052f8b0ef0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.089545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41137e97886749c79dac68052f8b0ef0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.092562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.119876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d9ab-ee80-4dcd-a424-e7e1fe58bf54 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.140294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.141003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.141289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.141602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] PATH_INFO: `/v3/projects/4dac84ced24a4030991c3e7bc5854227` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.145677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dac84ced24a4030991c3e7bc5854227)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.147846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.178234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41137e97886749c79dac68052f8b0ef0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.181004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41137e97886749c79dac68052f8b0ef0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.191402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41137e97886749c79dac68052f8b0ef0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.192559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41137e97886749c79dac68052f8b0ef0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.212461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41137e97886749c79dac68052f8b0ef0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.242154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 854/1697] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:06 2022] DELETE /identity/v3/users/41137e97886749c79dac68052f8b0ef0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:07.242993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:07.244838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.245807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dac84ced24a4030991c3e7bc5854227', 'request_id': 'req-ef77613f-2ecf-4805-be9d-f77c422b42ae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.256924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dac84ced24a4030991c3e7bc5854227', 'request_id': 'req-ef77613f-2ecf-4805-be9d-f77c422b42ae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.267173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dac84ced24a4030991c3e7bc5854227', 'request_id': 'req-ef77613f-2ecf-4805-be9d-f77c422b42ae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.281566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] The token cache is being invalidate because project 4dac84ced24a4030991c3e7bc5854227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:07.281954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.282742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef77613f-2ecf-4805-be9d-f77c422b42ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.288185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 844/1698] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:06 2022] DELETE /identity/v3/projects/4dac84ced24a4030991c3e7bc5854227 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:07.288185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.288185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.288185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.328392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:07.330790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.332252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1d12-28c6-4fdd-a1a4-2c4423bc0b09 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.338226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.340002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.340379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.340743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] PATH_INFO: `/v3/projects/a4b8ea15e151462e98a707cead3768a1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.342545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 845/1699] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:07.342662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b8ea15e151462e98a707cead3768a1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.343744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:07.343876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 07:24:07.343876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Sep 28 07:24:07.343876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: Sep 28 07:24:07.345433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.345562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.363432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdedf4-b2be-452a-adf3-26259c3c3729 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.370046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.371209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.371884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.372373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] PATH_INFO: `/v3/users/34991d45201c43938b347a4e8c91502c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.373785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=34991d45201c43938b347a4e8c91502c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.376770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.423052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:07.441932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b8ea15e151462e98a707cead3768a1', 'request_id': 'req-437f627d-8f7a-4885-a9eb-1863a716e03e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.448759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34991d45201c43938b347a4e8c91502c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.452144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34991d45201c43938b347a4e8c91502c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.452957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34991d45201c43938b347a4e8c91502c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.455911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b8ea15e151462e98a707cead3768a1', 'request_id': 'req-437f627d-8f7a-4885-a9eb-1863a716e03e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.466858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34991d45201c43938b347a4e8c91502c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.471652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b8ea15e151462e98a707cead3768a1', 'request_id': 'req-437f627d-8f7a-4885-a9eb-1863a716e03e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.485894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a3048151-4a1d-4f26-a1d9-80a43bb2a50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34991d45201c43938b347a4e8c91502c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.499154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] The token cache is being invalidate because project a4b8ea15e151462e98a707cead3768a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:07.500407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.501356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:07.503423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 855/1700] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:07 2022] DELETE /identity/v3/projects/a4b8ea15e151462e98a707cead3768a1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:07.504488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:07.508024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 846/1701] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:07 2022] DELETE /identity/v3/users/34991d45201c43938b347a4e8c91502c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:07.509613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.521538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7878b27-e710-48f2-8ac2-09245cf5683b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.521843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7878b27-e710-48f2-8ac2-09245cf5683b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.522115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7878b27-e710-48f2-8ac2-09245cf5683b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.611337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c7878b27-e710-48f2-8ac2-09245cf5683b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:07.612042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f627d-8f7a-4885-a9eb-1863a716e03e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.613919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c7878b27-e710-48f2-8ac2-09245cf5683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.750296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:07.753785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.757148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.757598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.757964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.763958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.813945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 847/1702] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:07.824939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.825460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.827915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.865906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:07.867610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:07.890344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 848/1703] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:07.895741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:07.916093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d41df-6562-45c5-b953-d5bed1a18324 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.926257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.927509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.927894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.928279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.929069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:07.929491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.931762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.964640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 849/1704] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:07.969551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.971620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:07.973828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:07.987895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deecae7-c92d-4946-bee3-488cda1a7446 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:07.993149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:07.993763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:07.994094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:07.994369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:07.995284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:07.998412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.000221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 856/1705] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:08.045929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 850/1706] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:07 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:08.047480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.058478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200c2e-7953-4104-9955-79c7cd63e6f7 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.121072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:08.183332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.185613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.186894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.187373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.188244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.190560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.223510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 857/1707] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:08 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:08.227877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.248413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f524-fa6f-4075-8220-1390ddd037ea admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.257277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.258299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.258777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.260437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.260437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.262101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.280853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 851/1708] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:08.285335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.306680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68776e-078a-4248-b1d1-6de3d124f23c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.320339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.321425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.322482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.322982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.323776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.327949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc257c-2560-4302-81d2-28242cf58a5a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.346453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 858/1709] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:08.351651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.365773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679734-a1c6-4c03-a619-bd3c9261d4d6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.374563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.375399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.375811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.376224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] PATH_INFO: `/v3/projects/c797d626ea3344f2b2b3d55d7b66df2c/users/550968105ffe45b89f714917dc02b983/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.417227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c797d626ea3344f2b2b3d55d7b66df2c, user_id=550968105ffe45b89f714917dc02b983, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.423252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.474693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 852/1710] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:08 2022] PUT /identity/v3/projects/c797d626ea3344f2b2b3d55d7b66df2c/users/550968105ffe45b89f714917dc02b983/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:08.490141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.490434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.490714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.554307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] MFA Rules not processed for user `550968105ffe45b89f714917dc02b983`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:08.554307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:08.665627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 859/1711] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:08.670661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.676998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.680323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:08.816145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581c38c-a64d-4873-8f6b-ecea0d7866a5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.824410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.824873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.825979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.826396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.827143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.829640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.843142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c416b9-22e4-421c-8db1-cea505154559 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:08.850117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:08.850780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:08.851058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:08.851331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] PATH_INFO: `/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:08.854160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d0e074e9c845aca8a38bfcd902f6ec)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:08.857290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:08.886301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 853/1712] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:08 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:08.887272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:08.942793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:08.947837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d0e074e9c845aca8a38bfcd902f6ec', 'request_id': 'req-7e29a875-a167-4131-becd-245b672037b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:08.960490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d0e074e9c845aca8a38bfcd902f6ec', 'request_id': 'req-7e29a875-a167-4131-becd-245b672037b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:08.980647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d0e074e9c845aca8a38bfcd902f6ec', 'request_id': 'req-7e29a875-a167-4131-becd-245b672037b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:08.996149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] The token cache is being invalidate because project a3d0e074e9c845aca8a38bfcd902f6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:08.996149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:08.996912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e29a875-a167-4131-becd-245b672037b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:09.001217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 860/1713] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:08 2022] DELETE /identity/v3/projects/a3d0e074e9c845aca8a38bfcd902f6ec => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:09.004527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947884-813a-4481-839d-1d16ffc9adec None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.004805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947884-813a-4481-839d-1d16ffc9adec None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.005117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947884-813a-4481-839d-1d16ffc9adec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.005874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:09.030605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddede1-5fa1-47af-90df-0f8581cceb2e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.051629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-23947884-813a-4481-839d-1d16ffc9adec None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:09.052982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-23947884-813a-4481-839d-1d16ffc9adec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:09.173264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.173899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.175177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.175465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.176120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.179340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.196992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 861/1714] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:09.204220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.204518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.204799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.244604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 854/1715] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:08 2022] POST /identity/v3/users => generated 293 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:09.250326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.261884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:09.274234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:09.275623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:09.297201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba6d5e-74ee-4ac4-a87b-36e7da6af146 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.306847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 862/1716] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:09.310237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.318110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.318738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.319020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.319294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.319952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.321607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.336726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec865f8-3e20-428b-8b77-89a92f0ce366 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.353506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.354158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.354583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.362750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.363792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:09.365222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.367229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 855/1717] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:09.368398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.369608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.395147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99f77-9c7a-459a-be2e-a41580bbce5f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.404019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 863/1718] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:09.407030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.408042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.408696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.409001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.409283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] PATH_INFO: `/v3/projects/e9c18fd129cd47d382567ea203d0b4e8/users/2b0b5e8de71342a19600913dd9270bd2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.432103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc0acf-dd5c-41fd-9c7a-0fd5c43532ea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.459675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.460312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.461189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.461472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.462118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.466681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c18fd129cd47d382567ea203d0b4e8, user_id=2b0b5e8de71342a19600913dd9270bd2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.469582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.471430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.521123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 856/1719] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:09 2022] PUT /identity/v3/projects/e9c18fd129cd47d382567ea203d0b4e8/users/2b0b5e8de71342a19600913dd9270bd2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:09.528293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.536287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:09.576597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 864/1720] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:09 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:09.580262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.715819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012111-6100-42df-83a8-d9674a5fb88c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.740751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2bda2-41ca-41ad-a0c2-ce923ca00ca1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.755990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.756613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.756906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.757189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.758428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.760183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.788465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 865/1721] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:09.789375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:09.807183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfd58-7c34-421f-ad34-d6a5047d007a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:09.828648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.829369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.829649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.829922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.830569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.832676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a678d20-ec09-4a9d-8c24-7dbde753e3de admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.850616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:09.862781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:09.863538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.865867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.866839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] PATH_INFO: `/v3/projects/776c9f40cffa41ddbc8d35a95627504a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.870391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=776c9f40cffa41ddbc8d35a95627504a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:09.872727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:09.882732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 866/1722] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:09 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:09.884396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:09.884690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:09.885011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:09.934331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:09.955191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] MFA Rules not processed for user `269ef35368734dcd9d3cc901db39fe10`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:09.956683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.008766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776c9f40cffa41ddbc8d35a95627504a', 'request_id': 'req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:10.019331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776c9f40cffa41ddbc8d35a95627504a', 'request_id': 'req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:10.037342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776c9f40cffa41ddbc8d35a95627504a', 'request_id': 'req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:10.082082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] The token cache is being invalidate because project 776c9f40cffa41ddbc8d35a95627504a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:10.082503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:10.084518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:10.085872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 867/1723] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:10.090556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 857/1724] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:09 2022] DELETE /identity/v3/projects/776c9f40cffa41ddbc8d35a95627504a => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:10.091664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:10.091985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:10.098542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.123068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed732a-9be4-4072-ae9b-36ddad7a0333 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:10.232171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:10.232937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.233055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.233860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] PATH_INFO: `/v3/projects/e9c18fd129cd47d382567ea203d0b4e8/users/2b0b5e8de71342a19600913dd9270bd2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.236895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c18fd129cd47d382567ea203d0b4e8, user_id=2b0b5e8de71342a19600913dd9270bd2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:10.245291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4266cb2-563e-4f07-b466-228ee9c3318f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:10.269515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 868/1725] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:10 2022] PUT /identity/v3/projects/e9c18fd129cd47d382567ea203d0b4e8/users/2b0b5e8de71342a19600913dd9270bd2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:10.269919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13c84-7cbe-4d17-9115-c3e4fdbcf2fc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:10.270911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.271168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.271419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.359538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:10.369679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.489400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 869/1726] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:10.492874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:10.494584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.506880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:10.507507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.508472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.508761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.509454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:10.511071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab3113-34fd-437a-af66-e281379b9659 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:10.531122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 858/1727] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:10.536093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5e74-6e96-42f1-ba61-aad0221bc0ee None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.536381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5e74-6e96-42f1-ba61-aad0221bc0ee None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.536661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5e74-6e96-42f1-ba61-aad0221bc0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.571462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4a6b5e74-6e96-42f1-ba61-aad0221bc0ee None None] MFA Rules not processed for user `2b0b5e8de71342a19600913dd9270bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:10.573876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4a6b5e74-6e96-42f1-ba61-aad0221bc0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.632299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:10.639255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:10.763332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5af6d6-00f7-443b-aebd-f10294dc18d8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:10.784116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:10.785856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.786145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.786416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.791663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.807523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 859/1728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:10.808824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.809137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.820899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.873679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:10.878543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:10.903780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 860/1729] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:10.907421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:10.933782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d901e48-0b6c-4e18-9d1a-2348520d720c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:10.949763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:10.950593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:10.950876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:10.951149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:10.960976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:10.964301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:10.973979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:10.978808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:10.998737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 861/1730] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:10.999611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.023339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:11.024259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 870/1731] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:11.025597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.061247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8441-00cd-4619-b787-6c409878d758 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.083561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.086007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.086298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.086570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.087270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:11.087523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.094926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.130059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 871/1732] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:11.131189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.169810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad51dd-a9a9-4e2a-98ee-20a803f3305c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.181997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.182690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.182980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.183643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] PATH_INFO: `/v3/projects/09a1b435d6e54b55921b06ad5abe315d/users/77b9909c7cb7491e9dd08c89eafd4aec/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.194300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051fe5f-fc5c-4ce9-b6f3-6d62019cc441 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.201727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.202401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.202678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.202996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.220614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.224709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78477773-341e-40e1-8eb1-8796e9855029 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.227389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a1b435d6e54b55921b06ad5abe315d, user_id=77b9909c7cb7491e9dd08c89eafd4aec, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.230351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.259819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 872/1733] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:11 2022] PUT /identity/v3/projects/09a1b435d6e54b55921b06ad5abe315d/users/77b9909c7cb7491e9dd08c89eafd4aec/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:11.260772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.291484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 862/1734] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:11.300653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.303325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.321822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.393244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] MFA Rules not processed for user `77b9909c7cb7491e9dd08c89eafd4aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:11.394620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:11.439475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1786d3-aad6-477a-9864-94841894943f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.469718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.470358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.470641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.470916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.471728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.473708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.552974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 873/1735] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:11 2022] POST /identity/v3/projects => generated 405 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:11.557694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.616509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 863/1736] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:11.618112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.632781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012aff-a09e-4f45-a37d-0e9c0ab24747 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.644215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:11.679185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:11.740158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.741544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.746274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.746760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.748765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.753713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.808433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac58fa-25eb-4397-856a-7bc1d73d7e8b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:11.817145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 864/1737] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:11 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:11.823603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.835417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:11.838860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:11.839158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:11.839440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:11.844277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:11.847577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:11.913734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 874/1738] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:11 2022] POST /identity/v3/users => generated 303 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:11.915316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:11.944411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa870a0-d7e9-403b-90b2-4239ca0476b3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.008059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e395c1-4d0a-4b54-ab14-2017805ef689 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.022185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.022811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.023090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.023432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.023998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.025922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.092933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 865/1739] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:11 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:12.096794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.117637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd46c-2bc6-4e69-aaf0-759b4b5564e2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.132081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.132081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.132081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.132081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.132416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.133985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.136431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.142344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.143210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.143707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.144601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.146666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.158513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 875/1740] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:12.158888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 866/1741] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:12.159889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.162118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.174272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a402e0-f651-4c08-bfe4-781a31c2e7d7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.174633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:12.179928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.181246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.181706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.182123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.183135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.185972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.199500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547b812-8bad-4d7e-bce8-45e1de044def admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.207395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 867/1742] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:12.208563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.211142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.212067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.212536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.213096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] PATH_INFO: `/v3/users/550968105ffe45b89f714917dc02b983` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.214721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=550968105ffe45b89f714917dc02b983)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.221496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.236252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9e8e-e4e0-4bbd-aeda-275f36bc31a2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.249073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.250085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.251709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.252299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.254276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.258972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.289689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 868/1743] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:12.289689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.307927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54f6c-b0f9-4b6d-9e39-4c1f55741d0f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.331172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.332657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.334392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.334808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] PATH_INFO: `/v3/projects/cf46fc260d5c4e4ebd4da468e9076f48/users/75c0be987392446486dea2bb9e853e6d/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.336682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550968105ffe45b89f714917dc02b983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:12.339283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550968105ffe45b89f714917dc02b983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:12.358026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550968105ffe45b89f714917dc02b983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:12.359564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550968105ffe45b89f714917dc02b983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:12.376963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf46fc260d5c4e4ebd4da468e9076f48, user_id=75c0be987392446486dea2bb9e853e6d, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.380808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.390440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c4179dcc-3557-43e7-8e17-fc5a21f89e2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550968105ffe45b89f714917dc02b983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:12.413895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 869/1744] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:12 2022] PUT /identity/v3/projects/cf46fc260d5c4e4ebd4da468e9076f48/users/75c0be987392446486dea2bb9e853e6d/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:12.414737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.420372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:12.440616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 876/1745] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:12 2022] DELETE /identity/v3/users/550968105ffe45b89f714917dc02b983 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:12.442367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8268fb1-3eba-4fc2-90f9-9284ec851400 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.442660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8268fb1-3eba-4fc2-90f9-9284ec851400 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.442985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8268fb1-3eba-4fc2-90f9-9284ec851400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.478532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f8268fb1-3eba-4fc2-90f9-9284ec851400 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:12.479897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f8268fb1-3eba-4fc2-90f9-9284ec851400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:12.493113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:12.574143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:12.618348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f44b2b-8f8f-4cd2-8215-15339f23f2a2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:12.684248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 877/1746] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:12.686443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.686729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.687001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.750144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:12.751380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:12.783059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 878/1747] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:12.783244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.784622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:12.797640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:12.800133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:12.800417 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:12.800693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] PATH_INFO: `/v3/projects/76548fc0f6af46d5871e4b1f8cefc8f2/users/92db3dcef7cf404fa06adfb3f5f6b63f/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:12.842452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76548fc0f6af46d5871e4b1f8cefc8f2, user_id=92db3dcef7cf404fa06adfb3f5f6b63f, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:12.847221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:12.887088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 870/1748] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:12 2022] PUT /identity/v3/projects/76548fc0f6af46d5871e4b1f8cefc8f2/users/92db3dcef7cf404fa06adfb3f5f6b63f/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:12.887088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:12.892771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:13.010171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f6169-6bc3-4d55-b2e1-dc79df6b1425 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:13.120692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da556c05-f3ff-476c-8e02-dbcdbb0b2c01 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:13.140821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:13.141661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:13.144517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:13.149953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:13.298423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:13.299066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:13.299391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:13.299677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:13.424022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:13.431779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b94b8e-00a9-4032-889f-4bb70bf70518 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:13.472144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 871/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:13.473547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:13.473836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:13.479345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:13.509100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:13.510571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:13.555365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 879/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:13.582444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] MFA Rules not processed for user `92db3dcef7cf404fa06adfb3f5f6b63f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:13.583998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:13.627675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:13.767255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 872/1751] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:13.774083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:13.792861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc61911-ac3d-4b1c-9cc9-095cfb2995ed service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:13.802545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:13.803186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:13.803803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:13.803803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:13.804364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:13.806921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734423c-00a5-4fa5-b379-36421760298c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:13.840181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:13.871263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:13.879308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 880/1752] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:13 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:13.888834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c93f8a-5d79-4976-9530-ecbbce117f65 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:14.038514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe983-6c01-4e65-b640-73c8d6fe6246 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:14.039063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe983-6c01-4e65-b640-73c8d6fe6246 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:14.039549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe983-6c01-4e65-b640-73c8d6fe6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:14.109718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:14.114083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:14.117308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:14.120856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:14.125177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:14.134287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:14.168015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-99efe983-6c01-4e65-b640-73c8d6fe6246 None None] MFA Rules not processed for user `d286b1e0afe9413291e0e2503b4c204a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:14.173851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99efe983-6c01-4e65-b640-73c8d6fe6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:14.301958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 873/1753] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:13 2022] POST /identity/v3/users => generated 299 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:14.303175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:14.355975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7f88d-60bd-49fc-b3f7-7ad260925da8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:14.378649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:14.383035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:14.387431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:14.387987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:14.389595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:14.392725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:14.417386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 874/1754] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:14.426079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:14.444891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf7a27-9f6d-416c-8608-146a28566e4b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:14.476653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:14.477696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:14.478121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:14.479790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] PATH_INFO: `/v3/projects/f537ee19c67a44c4a9bbf92408c335c9/users/b8c6dad02377424a9723a340164d3f87/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:14.493667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 881/1755] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:14.508975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:14.518735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:14.519376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:14.610630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f537ee19c67a44c4a9bbf92408c335c9, user_id=b8c6dad02377424a9723a340164d3f87, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:14.619087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:14.686631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] MFA Rules not processed for user `d286b1e0afe9413291e0e2503b4c204a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:14.705821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:14.710267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 875/1756] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:14 2022] PUT /identity/v3/projects/f537ee19c67a44c4a9bbf92408c335c9/users/b8c6dad02377424a9723a340164d3f87/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:14.713897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:14.913531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 882/1757] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:14.933487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:14.935545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:14.976111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fe594-9cbc-4703-b002-f6d00c5914fc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:15.002823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:15.003445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.003724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.003992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.004819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:15.007817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:15.037166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 876/1758] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:15.040915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:15.065377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5057f-d8ba-4ff3-bf5c-3c4a4348bbcf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:15.070629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f296fa-fb4a-4e16-b531-8efa85cad73a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:15.072134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:15.072874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.073205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.074471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.084994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:15.085955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.086366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.086938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] PATH_INFO: `/v3/projects/f537ee19c67a44c4a9bbf92408c335c9/users/b8c6dad02377424a9723a340164d3f87/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.093161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f537ee19c67a44c4a9bbf92408c335c9, user_id=b8c6dad02377424a9723a340164d3f87, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:15.100008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:15.102460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:15.102642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee59ed-8062-4b91-bb2f-a8e9388f40f9 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:15.115518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:15.130652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 883/1760] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:15.139426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 877/1760] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:15 2022] PUT /identity/v3/projects/f537ee19c67a44c4a9bbf92408c335c9/users/b8c6dad02377424a9723a340164d3f87/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:15.140272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da610-a5c7-4d5e-878e-f8f51bfeb13a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.140567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da610-a5c7-4d5e-878e-f8f51bfeb13a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.140842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da610-a5c7-4d5e-878e-f8f51bfeb13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.209824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:15.220551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b32da610-a5c7-4d5e-878e-f8f51bfeb13a None None] MFA Rules not processed for user `b8c6dad02377424a9723a340164d3f87`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:15.223681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b32da610-a5c7-4d5e-878e-f8f51bfeb13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:15.423547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 884/1761] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:15.435941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:15.439095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:15.612321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a784e-6e71-495e-8197-f83ad2c13a87 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:15.628144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:15.631356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.631751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.632104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.651992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:15.678989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef727b95-a3b3-4849-8b78-603d96a440bb service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:15.705211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 878/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:15.946570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:15.947068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:15.947141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:15.986512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] MFA Rules not processed for user `c0983c1e9f7649c2890000959cea452b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:15.988154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:16.178541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 885/1763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:16.195091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ad7e-e42b-4a06-af87-d2b5d9bf11b9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:16.195593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ad7e-e42b-4a06-af87-d2b5d9bf11b9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:16.195827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ad7e-e42b-4a06-af87-d2b5d9bf11b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:16.274130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0545ad7e-e42b-4a06-af87-d2b5d9bf11b9 None None] MFA Rules not processed for user `c0983c1e9f7649c2890000959cea452b`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:16.281522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0545ad7e-e42b-4a06-af87-d2b5d9bf11b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:16.299286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 879/1764] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:16.310844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:16.312250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:16.499954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf5d47-9e3b-439b-a701-cdcbcb265daa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:16.511498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:16.512191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:16.512500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:16.512878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:16.539411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:16.542382 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20fe9-378d-401a-93ee-03eb155782fe service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:16.573173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 886/1765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:16.714991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:16.715513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:16.717302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:16.722279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 880/1766] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 07:24:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:16.735575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:16.735864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:16.736207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:16.799597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:16.849017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:16.853448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:16.977336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:16.979635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.008837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 887/1767] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:24:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:17.009785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:17.011188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.024476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ab75-4bf5-4732-b784-15591c71c2db None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:17.028366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:17.029455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:17.029755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:17.030030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:17.032046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.161353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43af0-aa0c-4c70-93d5-d942a140442f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:17.168790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:17.169234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:17.170078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:17.170187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:17.170437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] PATH_INFO: `/v3/projects/c797d626ea3344f2b2b3d55d7b66df2c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:17.171696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c797d626ea3344f2b2b3d55d7b66df2c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:17.172933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:17.173736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:17.197450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 888/1768] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:17.202176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:17.203896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.266285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c03db-85ad-4d57-bd93-d09a9cc4ffa9 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:17.275767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:17.276659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:17.277254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:17.277530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:17.290727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:17.292378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ff93a-22e8-43ed-bc82-c5aab3519970 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:17.305770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 889/1769] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:17.318494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c797d626ea3344f2b2b3d55d7b66df2c', 'request_id': 'req-81b85475-1810-4bcd-8d93-fd050583ff98'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.328794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c797d626ea3344f2b2b3d55d7b66df2c', 'request_id': 'req-81b85475-1810-4bcd-8d93-fd050583ff98'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.361258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cec6-4d86-4d9e-9be7-64de1e1c6105 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:17.361684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cec6-4d86-4d9e-9be7-64de1e1c6105 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:17.362137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cec6-4d86-4d9e-9be7-64de1e1c6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:17.373644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c797d626ea3344f2b2b3d55d7b66df2c', 'request_id': 'req-81b85475-1810-4bcd-8d93-fd050583ff98'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.402306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] The token cache is being invalidate because project c797d626ea3344f2b2b3d55d7b66df2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:17.405229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.406874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.417960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 881/1770] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:16 2022] DELETE /identity/v3/projects/c797d626ea3344f2b2b3d55d7b66df2c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:17.421176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:17.425692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.445870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-93b4cec6-4d86-4d9e-9be7-64de1e1c6105 None None] MFA Rules not processed for user `764cfe23613e4a97ba05f8bf0ed77ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:17.450842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93b4cec6-4d86-4d9e-9be7-64de1e1c6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:17.516043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:17.665400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85475-1810-4bcd-8d93-fd050583ff98 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:17.673472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:17.677101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:17.677422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:17.677693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] PATH_INFO: `/v3/users/2b0b5e8de71342a19600913dd9270bd2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:17.680735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0b5e8de71342a19600913dd9270bd2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:17.685824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:17.730265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 890/1771] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:17.793406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0b5e8de71342a19600913dd9270bd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.795793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0b5e8de71342a19600913dd9270bd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.796482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0b5e8de71342a19600913dd9270bd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.815379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0b5e8de71342a19600913dd9270bd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.838333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-fcbae3f6-4f8a-4262-a1e2-6c9f9ec13e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0b5e8de71342a19600913dd9270bd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:17.866723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 882/1772] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:17 2022] DELETE /identity/v3/users/2b0b5e8de71342a19600913dd9270bd2 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:18.089647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757bda-74e0-4a89-916c-3fe63030323c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.090190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757bda-74e0-4a89-916c-3fe63030323c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.090190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757bda-74e0-4a89-916c-3fe63030323c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.093230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 891/1773] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 07:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:18.098893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.099182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.099646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.150051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:18.151312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.398882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.398882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.398882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.447910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 883/1774] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:24:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:18.460284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] MFA Rules not processed for user `75c0be987392446486dea2bb9e853e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:18.462339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.484116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:18.486041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.498943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:18.503032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36c7c2-8f8e-4912-9ba8-1b8677c082dc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:18.512146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:18.518958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.519232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.519708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.552986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:18.556521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:18.577964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 884/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:18.580277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:18.583097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.648831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 892/1776] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:18 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:18.652094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:18.656185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.706218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:18.774373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778ba3-9611-403e-939b-cb9fdd191772 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:18.786797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:18.787454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.787731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.787998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] PATH_INFO: `/v3/projects/e9c18fd129cd47d382567ea203d0b4e8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.789163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c18fd129cd47d382567ea203d0b4e8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:18.791346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:18.830796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fd778-c6a0-4345-a9b2-1ea8bbbbe7da None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:18.840093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:18.841190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:18.841768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:18.842213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:18.843010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:18.855229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:18.936079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 893/1777] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:18 2022] POST /identity/v3/projects => generated 405 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:18.936979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:18.938447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c18fd129cd47d382567ea203d0b4e8', 'request_id': 'req-45c8bef7-2af9-416d-9c39-74ca58b5ab44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:18.946335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:18.947562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c18fd129cd47d382567ea203d0b4e8', 'request_id': 'req-45c8bef7-2af9-416d-9c39-74ca58b5ab44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:18.966662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c18fd129cd47d382567ea203d0b4e8', 'request_id': 'req-45c8bef7-2af9-416d-9c39-74ca58b5ab44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:18.984056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] The token cache is being invalidate because project e9c18fd129cd47d382567ea203d0b4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:18.985826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:18.988681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:18.994307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 885/1778] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:18 2022] DELETE /identity/v3/projects/e9c18fd129cd47d382567ea203d0b4e8 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:18.994307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:18.997918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.167260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08b55a-8ae0-4873-82e4-ec550929ba95 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:19.168176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8bef7-2af9-416d-9c39-74ca58b5ab44 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:19.186121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:19.187030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.187489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.187935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.192244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.297231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:19.314080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:19.314713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.315067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.315341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.318188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.353488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:19.355235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:19.355825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:19.363449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2786b4-40b3-4cb8-989d-f3467766914b service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:19.371126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 894/1779] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:19.372193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:19.373942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.394329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:19.426811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 886/1780] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:19.430525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f275-a2c1-403e-9100-2f91efe1844b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.438674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f275-a2c1-403e-9100-2f91efe1844b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.439000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f275-a2c1-403e-9100-2f91efe1844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.554189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2392f275-a2c1-403e-9100-2f91efe1844b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:19.558453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2392f275-a2c1-403e-9100-2f91efe1844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.586931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7101-c5df-4c50-9468-2004f0a218bd service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:19.587634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 887/1781] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:19.604093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.604548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.606977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.613821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:19.616450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.616752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.623286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.624064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:19.631395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e225-f3b6-46e6-bff2-db3ac4b31f07 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:19.646995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] MFA Rules not processed for user `269ef35368734dcd9d3cc901db39fe10`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:19.648607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.709324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 895/1782] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:19 2022] POST /identity/v3/users => generated 309 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:19.714254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.721231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.723292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.768549 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:19.770241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:19.803518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 896/1783] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:19.808545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:19.833595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2445d0-2152-4521-bb12-2a341084f359 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:19.864195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:19.865802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.866086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.868672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.871756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:19.877150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31cc8f-5b06-45ae-b6c8-d5e38868483b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:19.890179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 888/1784] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:19.896429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:19.933201 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 897/1785] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:19.937309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.937653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.939662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.943535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671ff9d-3939-4bfb-a2d5-97f53a1147bb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:19.957317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:19.958009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:19.960486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:19.960821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:19.961532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:19.965177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:19.989410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] MFA Rules not processed for user `d286b1e0afe9413291e0e2503b4c204a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:19.989720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:19.990463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:20.050653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 889/1786] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:20.051767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.075411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a025a-f058-404a-8f7c-110acf0835ec admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.093288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.094154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.096071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.096910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] PATH_INFO: `/v3/projects/6e9cea1cdb304a40a4707a236bd192c4/users/7ef9f03bf3c14982a8b008d316804296/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.134905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 898/1787] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:20.135779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.157025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9cea1cdb304a40a4707a236bd192c4, user_id=7ef9f03bf3c14982a8b008d316804296, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.162684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5ca97-5c12-48eb-bfdd-71b732691d34 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.172967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.174926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.175569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.175852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.176717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.177579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.179497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.212383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 890/1788] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:20 2022] PUT /identity/v3/projects/6e9cea1cdb304a40a4707a236bd192c4/users/7ef9f03bf3c14982a8b008d316804296/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:20.216880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.271691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 899/1789] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:20 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:20.273087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.363459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:20.449711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cfc5-fccc-4586-b6ef-d993ee051fc5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.451534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a5e8-706a-45b3-a0d9-640fe997c1c2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.470155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.472274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.472699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.473002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.473278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.473961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.476014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.476297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.476575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.483325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.484230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.486859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.534659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 891/1790] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:20.536647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.569627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 900/1791] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:20 2022] POST /identity/v3/users => generated 304 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:20.576870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.595596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea0b4-ff08-4818-a60c-33a84526c3d0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.626953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.627649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76c72-a2c8-421b-987f-eb874ffef001 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:20.629239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.631356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.632601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] PATH_INFO: `/v3/projects/6e9cea1cdb304a40a4707a236bd192c4/users/7ef9f03bf3c14982a8b008d316804296/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.641197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9cea1cdb304a40a4707a236bd192c4, user_id=7ef9f03bf3c14982a8b008d316804296, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.650483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.651918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:20.652498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:20.652778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:20.653710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:20.654573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:20.656198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:20.704266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 901/1792] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:20.705176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:20.721328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 892/1793] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:20 2022] PUT /identity/v3/projects/6e9cea1cdb304a40a4707a236bd192c4/users/7ef9f03bf3c14982a8b008d316804296/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:20.765533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee519ffc-167d-4853-8aa8-394f793d4d17 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:21.015230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:21.015993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:21.016469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:21.016753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:21.017563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:21.027818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefffa8-46ea-4552-8e66-8e65961526e4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:21.080485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:21.081081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 902/1794] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:21.100643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aa20-a4d7-49ef-9b98-f78e8dee57df admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:21.116470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:21.117273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:21.117913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:21.118198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] PATH_INFO: `/v3/projects/6570d1b5ea13405ea22570e38f080851/users/3d2341b48e424ccabfed2ecdd49b5fac/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:21.168348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6570d1b5ea13405ea22570e38f080851, user_id=3d2341b48e424ccabfed2ecdd49b5fac, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:21.177205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:21.228886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 893/1795] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:21 2022] PUT /identity/v3/projects/6570d1b5ea13405ea22570e38f080851/users/3d2341b48e424ccabfed2ecdd49b5fac/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:21.233989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:21.234511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:21.234880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:21.354143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] MFA Rules not processed for user `3d2341b48e424ccabfed2ecdd49b5fac`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:21.366104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:21.574192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 903/1796] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:21 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:21.579039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:21.633634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:21.776801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa129b-43c6-433d-b0ed-965abb83856e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:21.786262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:21.787135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:21.787453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:21.787743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:21.788707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:21.802793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:21.904968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 894/1797] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:21 2022] POST /identity/v3/projects => generated 409 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:21.908866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:21.924839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b094027-cf86-44a3-9b20-e514774000ad None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:22.091854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:22.181729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:22.182368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.182649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.182917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.183577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:22.190443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:22.274645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 904/1798] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:21 2022] POST /identity/v3/users => generated 311 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:22.275692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:22.296764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d68-1171-40a2-9935-bb6032f83138 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:22.306009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:22.306991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.307473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.307826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.308862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:22.312501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:22.331537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 895/1799] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:22.335363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:22.352878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc6d2-4e50-4a17-99d0-0f22ba7a0d0e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:22.371172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:22.372599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.372759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.373306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] PATH_INFO: `/v3/projects/f75b61013ed44d75bc44951b0f7171b8/users/a03bce60a10244a2a579118122e8cd87/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.419502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75b61013ed44d75bc44951b0f7171b8, user_id=a03bce60a10244a2a579118122e8cd87, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:22.422243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:22.461155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 905/1800] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:22 2022] PUT /identity/v3/projects/f75b61013ed44d75bc44951b0f7171b8/users/a03bce60a10244a2a579118122e8cd87/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:22.465703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:22.479083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a16ec4-a5ed-494b-8306-df498eed826a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:22.677608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:22.691002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.691614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.692083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.693029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:22.694885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d1a5e-5a6b-40b4-bfe8-e9017b3eb741 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:22.726374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:22.738542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 896/1801] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:22.742694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957012-f8f4-4c44-a2a6-fa6ef1500c27 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:22.759907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:22.761022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.761533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.762019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] PATH_INFO: `/v3/projects/f75b61013ed44d75bc44951b0f7171b8/users/a03bce60a10244a2a579118122e8cd87/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.765127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75b61013ed44d75bc44951b0f7171b8, user_id=a03bce60a10244a2a579118122e8cd87, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:22.768426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:22.833571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 906/1802] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:22 2022] PUT /identity/v3/projects/f75b61013ed44d75bc44951b0f7171b8/users/a03bce60a10244a2a579118122e8cd87/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:22.834704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:22.834996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:22.835274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:22.907870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] MFA Rules not processed for user `a03bce60a10244a2a579118122e8cd87`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:22.909806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:23.153711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 897/1803] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:22 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:23.187498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:23.192871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:23.408368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1b60-8ab7-4d15-a743-73a0230cca61 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:23.414786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:23.416109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:23.416558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:23.417054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:23.419583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:23.520622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:23.522296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:23.543216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690d8a7-90be-4691-8932-bc2bf9acd7b1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:23.554658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:23.555871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:23.556298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:23.556731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:23.583826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:23.588901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:23.608517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 907/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:23.612509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:23.614196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab055-cb59-45bc-9d90-91173ef04cf7 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:23.682773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 898/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:23 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:24:24.109694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:24.123217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e78aec-a71f-44cf-9622-6a2855acfb9d service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:24.149586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:24.150649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:24.150948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:24.151215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:24.199008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff880-5bc5-44e2-b2a1-b17b448d68d4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:24.199289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff880-5bc5-44e2-b2a1-b17b448d68d4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:24.199560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff880-5bc5-44e2-b2a1-b17b448d68d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:24.208816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:24.256763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:24.277337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-340ff880-5bc5-44e2-b2a1-b17b448d68d4 None None] MFA Rules not processed for user `7ef9f03bf3c14982a8b008d316804296`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:24.282907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-340ff880-5bc5-44e2-b2a1-b17b448d68d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:24.306314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:24.307905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41259679-e061-4f24-9dff-c567334ebbc8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:24.319622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 908/1806] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:24 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:24.519148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 899/1807] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:24 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:24.566100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fb72-6bab-47c4-b54e-6852336ca2e6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:24.566560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fb72-6bab-47c4-b54e-6852336ca2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:24.566731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fb72-6bab-47c4-b54e-6852336ca2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:24.594865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-53d5fb72-6bab-47c4-b54e-6852336ca2e6 None None] MFA Rules not processed for user `764cfe23613e4a97ba05f8bf0ed77ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:24.596327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-53d5fb72-6bab-47c4-b54e-6852336ca2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:24.712865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 909/1808] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:25.638551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8443-788d-4262-acf3-58e5fc5e33e1 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:25.639475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8443-788d-4262-acf3-58e5fc5e33e1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:25.640063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8443-788d-4262-acf3-58e5fc5e33e1 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:25.641923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 900/1809] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:24:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:25.657758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:25.658310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:25.658661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:25.688888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:25.690390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:25.786828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 910/1810] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:24:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:25.808700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de8443-788d-4262-acf3-58e5fc5e33e1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:25.834406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de8443-788d-4262-acf3-58e5fc5e33e1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:25.841379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:25.842390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:25.842390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:25.842779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:25.852224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:25.853916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:25.865052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 901/1811] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:26.502200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:26.689562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1e30-5db2-4929-9896-db46e87accc6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:26.698914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:26.702375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:26.702952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:26.703368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] PATH_INFO: `/v3/users/75c0be987392446486dea2bb9e853e6d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:26.705013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c0be987392446486dea2bb9e853e6d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:26.707259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:26.738894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:26.798458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c0be987392446486dea2bb9e853e6d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:26.799323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c0be987392446486dea2bb9e853e6d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:26.812476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c0be987392446486dea2bb9e853e6d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:26.813558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c0be987392446486dea2bb9e853e6d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:26.836846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c0be987392446486dea2bb9e853e6d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:26.881584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 911/1812] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:26 2022] DELETE /identity/v3/users/75c0be987392446486dea2bb9e853e6d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:27.224619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:27.252050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8889a3-6dbc-4bcd-8447-226a38b4a2e1 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:27.393830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:27.394714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:27.395176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:27.395629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] PATH_INFO: `/v3/projects/cf46fc260d5c4e4ebd4da468e9076f48` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:27.397810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf46fc260d5c4e4ebd4da468e9076f48)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:27.400620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:27.514399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf46fc260d5c4e4ebd4da468e9076f48', 'request_id': 'req-61cdb496-fc92-4254-86d6-a5a85d7fafea'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.527210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf46fc260d5c4e4ebd4da468e9076f48', 'request_id': 'req-61cdb496-fc92-4254-86d6-a5a85d7fafea'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.547885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf46fc260d5c4e4ebd4da468e9076f48', 'request_id': 'req-61cdb496-fc92-4254-86d6-a5a85d7fafea'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.571807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] The token cache is being invalidate because project cf46fc260d5c4e4ebd4da468e9076f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:27.572108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.575028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.580046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 902/1813] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:27 2022] DELETE /identity/v3/projects/cf46fc260d5c4e4ebd4da468e9076f48 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:27.581177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:27.583350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:27.598795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30c71-2585-484a-829e-dd6165ae1695 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:27.670687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:27.671863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:27.672455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:27.672969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] PATH_INFO: `/v3/users/7ef9f03bf3c14982a8b008d316804296` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:27.674592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef9f03bf3c14982a8b008d316804296)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:27.676771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:27.781284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef9f03bf3c14982a8b008d316804296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.782695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef9f03bf3c14982a8b008d316804296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.792566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef9f03bf3c14982a8b008d316804296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.793779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef9f03bf3c14982a8b008d316804296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.811126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3c43a531-693c-4546-8751-240fb04dedbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef9f03bf3c14982a8b008d316804296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:27.838667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 912/1814] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:27 2022] DELETE /identity/v3/users/7ef9f03bf3c14982a8b008d316804296 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:28.234330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:28.236520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:28.253951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:28.262502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdb496-fc92-4254-86d6-a5a85d7fafea admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:28.392044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:28.392735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:28.393049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:28.393335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] PATH_INFO: `/v3/projects/6e9cea1cdb304a40a4707a236bd192c4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:28.394755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9cea1cdb304a40a4707a236bd192c4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:28.397288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:28.487680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9cea1cdb304a40a4707a236bd192c4', 'request_id': 'req-2dcbc975-b51c-4f87-bd37-66caa4838234'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:28.501154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9cea1cdb304a40a4707a236bd192c4', 'request_id': 'req-2dcbc975-b51c-4f87-bd37-66caa4838234'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:28.515559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9cea1cdb304a40a4707a236bd192c4', 'request_id': 'req-2dcbc975-b51c-4f87-bd37-66caa4838234'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:28.532740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] The token cache is being invalidate because project 6e9cea1cdb304a40a4707a236bd192c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:28.533064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:28.533800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2dcbc975-b51c-4f87-bd37-66caa4838234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:28.535634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 903/1815] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:28 2022] DELETE /identity/v3/projects/6e9cea1cdb304a40a4707a236bd192c4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:28.540772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:28.541258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:28.541921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:28.582162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:28.583883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:28.631000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:28.722461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 913/1816] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:28.730230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:28.730634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:28.731124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:28.766483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:28.768191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:28.832543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 904/1817] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:28.832628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:28.836056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c5e1-86a8-42b6-94dd-c733a0d72b37 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:28.840153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:28.840897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:28.841210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:28.841482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:28.842141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:28.842573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:28.844508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:28.868073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 914/1818] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:28.873759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:28.883880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56030fb1-c16a-4af0-a21a-b66359907128 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:28.889195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:28.889887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:28.890156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:28.890425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:28.891079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:28.893073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:28.933164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 905/1819] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:28 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:28.937129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:28.954779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c95e1-a7ca-4199-ae93-3b81a3b44ca8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.030856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.031475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.031755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.032029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.032692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.036687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.074444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 915/1820] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:28 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:29.077558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.096853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a299c-fa95-45f7-a17a-e978fd2b1f51 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.102826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.103533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.103884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.104239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.104971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.106866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.131261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 906/1821] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:29.132000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.144479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11f69-347d-47f4-9eb8-bdf5fc91c558 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.151000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.151620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.151894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.152165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.153026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.154876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e5819-6a57-4aed-8df6-a9ce414fe03b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.173397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.173837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 916/1822] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:29.186122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd837c-77ee-452e-9474-ba9b62e18843 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.191764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.192443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.192721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.193051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] PATH_INFO: `/v3/projects/2cfbae58e2d14e659c6e93cc39f64fe2/users/c88155549d79434fa91b72d8733cdb28/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.213040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfbae58e2d14e659c6e93cc39f64fe2, user_id=c88155549d79434fa91b72d8733cdb28, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.216308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.234392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 907/1823] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:29 2022] PUT /identity/v3/projects/2cfbae58e2d14e659c6e93cc39f64fe2/users/c88155549d79434fa91b72d8733cdb28/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:29.245969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.246258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.247104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.284585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] MFA Rules not processed for user `c88155549d79434fa91b72d8733cdb28`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:29.285433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:29.371773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 917/1824] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:29.373095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.413953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:29.448938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064baa-a197-4de1-9886-437f73937e14 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.453843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.454444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.454718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.454984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.455616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.457530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.486956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 908/1825] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:29 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:29.490969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.501781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0da5e-ce21-4083-8565-16ece610cd9c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.513586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:29.558498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.559149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.559661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.559927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.560553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.562510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.584226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 918/1826] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:29 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:29.588230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.599128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec3948-5526-4f28-b038-9b0afb5a3a1b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.603653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.604239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.604515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.604783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.605526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.607107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.617432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 909/1827] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:29.621756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.631765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a97187-4332-4627-81ee-4b85617b2c92 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.636343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.637032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.638065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.638374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] PATH_INFO: `/v3/projects/0b4e081954ad42c985b2b88a966f6030/users/5900f65df5cf4f00810d2217ff755b54/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.654664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4e081954ad42c985b2b88a966f6030, user_id=5900f65df5cf4f00810d2217ff755b54, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.657546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.671801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 919/1828] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:29 2022] PUT /identity/v3/projects/0b4e081954ad42c985b2b88a966f6030/users/5900f65df5cf4f00810d2217ff755b54/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:29.675924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.686584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67a13-5226-4bea-b919-f84d56ae4715 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.766010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.766010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.766010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.766010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.766547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.766547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe26b2a-b12c-4dc4-8417-e84641a3e89d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.771222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 910/1829] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:29.775893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.788066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1f0f7-4082-4785-a5d3-20b88f8fa93e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:29.792996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:29.794422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.794709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.794974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] PATH_INFO: `/v3/projects/0b4e081954ad42c985b2b88a966f6030/users/5900f65df5cf4f00810d2217ff755b54/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.797066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4e081954ad42c985b2b88a966f6030, user_id=5900f65df5cf4f00810d2217ff755b54, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:29.799729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:29.812693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 920/1830] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:29 2022] PUT /identity/v3/projects/0b4e081954ad42c985b2b88a966f6030/users/5900f65df5cf4f00810d2217ff755b54/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:29.823519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:29.823802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:29.824077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:29.854479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] MFA Rules not processed for user `5900f65df5cf4f00810d2217ff755b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:29.855689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:29.948998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 911/1831] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:29.967072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:29.973368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:30.049701 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912592a-5ccc-4e73-ad31-32f3b4d0cd82 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:30.056532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:30.058867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:30.058867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:30.058867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:30.068371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:30.069852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149da26-ddb2-4863-be2f-4aff2cf0da0f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:30.081716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 921/1832] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:30.165517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:30.166894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:30.179415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781529a6-12c6-4460-bff2-d963f3b54506 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:30.183325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:30.183920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:30.184325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:30.184602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:30.220291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:30.255108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:30.256542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:30.266077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 912/1833] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:30.459534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4df73-4819-4a22-9f2c-5288b8e70fe8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:30.460437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4df73-4819-4a22-9f2c-5288b8e70fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:30.460522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4df73-4819-4a22-9f2c-5288b8e70fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:30.484178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2de4df73-4819-4a22-9f2c-5288b8e70fe8 None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:30.485424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2de4df73-4819-4a22-9f2c-5288b8e70fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:30.570581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 922/1834] 10.222.0.33 () {58 vars in 1015 bytes} [Wed Sep 28 07:24:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:30.622589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:30.625004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:30.647574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11bc4-a6aa-442c-b291-46b587e7fc0a service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:30.652275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:30.653085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:30.653349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:30.653774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:30.663646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:30.666740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:30.716244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:30.717114 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:30.717392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:30.723579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 913/1835] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:30.774926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:30.807950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:30.810364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:30.951152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 923/1836] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:24:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:31.213242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:31.214900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:31.441580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b870-1f81-4482-987e-7e702d9d684c service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:31.456246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:31.456975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:31.457273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:31.457522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:31.483669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:31.485580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daad93b-6055-4fce-bf19-467ddad221b9 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:31.498949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 914/1837] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:24:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:24:31.873267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:31.873890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.029669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b4fcf-7905-419b-ba42-e90bf494f140 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:32.035978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:32.037010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.037430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.037834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] PATH_INFO: `/v3/users/77b9909c7cb7491e9dd08c89eafd4aec` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.039501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b9909c7cb7491e9dd08c89eafd4aec)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:32.042125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:32.125045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b9909c7cb7491e9dd08c89eafd4aec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:32.126244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b9909c7cb7491e9dd08c89eafd4aec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:32.137176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b9909c7cb7491e9dd08c89eafd4aec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:32.138429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b9909c7cb7491e9dd08c89eafd4aec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:32.159620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b9909c7cb7491e9dd08c89eafd4aec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:32.185859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 924/1838] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:31 2022] DELETE /identity/v3/users/77b9909c7cb7491e9dd08c89eafd4aec => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:32.191047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a2c-ea0c-4f47-a3ea-29e9d7a99bae None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.191171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a2c-ea0c-4f47-a3ea-29e9d7a99bae None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.191438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a2c-ea0c-4f47-a3ea-29e9d7a99bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.219215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-19d07a2c-ea0c-4f47-a3ea-29e9d7a99bae None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:32.220887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19d07a2c-ea0c-4f47-a3ea-29e9d7a99bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.339855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 915/1839] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:32.349960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:32.351609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.376525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:32.454290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0660d2-b521-41e0-8831-efdffe1dd884 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.454778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0660d2-b521-41e0-8831-efdffe1dd884 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.454778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0660d2-b521-41e0-8831-efdffe1dd884 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.456215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 916/1840] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:32.467771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8dcab-b27c-48d1-9c8b-fe2c6b14d8a7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:32.471326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:32.473515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.473515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.473515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.483114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.486074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.486074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.494274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:32.497551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:32.531327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:32.532593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.538512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 925/1841] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:32.576803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:32.648632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 917/1842] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:24:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:32.663951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:32.665814 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.700831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb78dc-c284-42f2-b89e-2be387fdb948 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:32.714343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:32.714980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.717129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.717276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.730347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:32.737735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:32.760053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 926/1843] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:32.945933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:32.947392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:32.963466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddfe20-889d-4803-b005-c451f07adde8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:32.970132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:32.970790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:32.971066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:32.971335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] PATH_INFO: `/v3/projects/09a1b435d6e54b55921b06ad5abe315d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:32.972472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a1b435d6e54b55921b06ad5abe315d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:32.974418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:33.031321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a1b435d6e54b55921b06ad5abe315d', 'request_id': 'req-c40444a5-3a09-4fb4-bda0-a27c348e4154'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.043266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a1b435d6e54b55921b06ad5abe315d', 'request_id': 'req-c40444a5-3a09-4fb4-bda0-a27c348e4154'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.052735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a1b435d6e54b55921b06ad5abe315d', 'request_id': 'req-c40444a5-3a09-4fb4-bda0-a27c348e4154'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.067306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] The token cache is being invalidate because project 09a1b435d6e54b55921b06ad5abe315d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:33.067589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.068287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.073479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 918/1844] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:32 2022] DELETE /identity/v3/projects/09a1b435d6e54b55921b06ad5abe315d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:33.075539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:33.086816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab4bed-aad9-49d1-9657-4aa9ee1b1ebe service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:33.150190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:33.150841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:33.151118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:33.151420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] PATH_INFO: `/v3/users/92db3dcef7cf404fa06adfb3f5f6b63f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:33.152669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92db3dcef7cf404fa06adfb3f5f6b63f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:33.155137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:33.205771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92db3dcef7cf404fa06adfb3f5f6b63f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.206434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92db3dcef7cf404fa06adfb3f5f6b63f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.213688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92db3dcef7cf404fa06adfb3f5f6b63f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.214389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92db3dcef7cf404fa06adfb3f5f6b63f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.226964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92db3dcef7cf404fa06adfb3f5f6b63f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.242636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 927/1845] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:33 2022] DELETE /identity/v3/users/92db3dcef7cf404fa06adfb3f5f6b63f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:33.571803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:33.575078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:33.592019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40444a5-3a09-4fb4-bda0-a27c348e4154 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:33.723312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:33.723974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:33.724248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:33.724525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] PATH_INFO: `/v3/projects/76548fc0f6af46d5871e4b1f8cefc8f2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:33.727270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76548fc0f6af46d5871e4b1f8cefc8f2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:33.734397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:33.792292 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:33.795803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76548fc0f6af46d5871e4b1f8cefc8f2', 'request_id': 'req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.802877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76548fc0f6af46d5871e4b1f8cefc8f2', 'request_id': 'req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.812176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76548fc0f6af46d5871e4b1f8cefc8f2', 'request_id': 'req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.823898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] The token cache is being invalidate because project 76548fc0f6af46d5871e4b1f8cefc8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:33.824263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.825585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:33.827420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 919/1846] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:33 2022] DELETE /identity/v3/projects/76548fc0f6af46d5871e4b1f8cefc8f2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:33.831357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:33.832761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:33.850892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24955-d7a0-48fa-9452-a25e80721209 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:33.869603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:33.946728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:33.947535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:33.947933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:33.948299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] PATH_INFO: `/v3/users/b8c6dad02377424a9723a340164d3f87` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:33.950799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c6dad02377424a9723a340164d3f87)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:33.953241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:34.035724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c6dad02377424a9723a340164d3f87'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.036410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c6dad02377424a9723a340164d3f87'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.044574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c6dad02377424a9723a340164d3f87'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.045373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c6dad02377424a9723a340164d3f87'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.061216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b72428de-e539-47d7-824c-920abecc0dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c6dad02377424a9723a340164d3f87'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.075962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 928/1847] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:33 2022] DELETE /identity/v3/users/b8c6dad02377424a9723a340164d3f87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:34.398375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:34.400172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:34.424737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5fcd-9c1b-467f-b488-b31fb9d7c8e9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:34.502190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:34.507964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:34.507964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:34.507964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] PATH_INFO: `/v3/projects/f537ee19c67a44c4a9bbf92408c335c9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:34.509366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f537ee19c67a44c4a9bbf92408c335c9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:34.511155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:34.570164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f537ee19c67a44c4a9bbf92408c335c9', 'request_id': 'req-1ffd50f7-b406-40e8-9372-66232b45b955'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.578068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f537ee19c67a44c4a9bbf92408c335c9', 'request_id': 'req-1ffd50f7-b406-40e8-9372-66232b45b955'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.586164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f537ee19c67a44c4a9bbf92408c335c9', 'request_id': 'req-1ffd50f7-b406-40e8-9372-66232b45b955'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.610107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] The token cache is being invalidate because project f537ee19c67a44c4a9bbf92408c335c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:34.610477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.611164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1ffd50f7-b406-40e8-9372-66232b45b955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:34.612686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 920/1848] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:34 2022] DELETE /identity/v3/projects/f537ee19c67a44c4a9bbf92408c335c9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:34.620606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:34.620606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:34.620985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:34.657866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:34.659227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:34.785990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:34.785990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:34.786320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:34.791404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 929/1849] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:34.819463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:34.821906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:34.846817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 921/1850] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:34.852335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:34.870792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed911c-258a-47a7-88a5-9fe96fbbc38b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:34.885598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:34.887658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:34.887957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:34.888937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:34.889713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:34.892845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:34.896185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:34.932002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:34.939736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 930/1851] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:34.944241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c88d3-a02f-4cb7-ab84-01f53e80a47f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:34.977637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:34.979437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:34.980546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:34.981429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:34.982109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:34.989096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:35.068608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 922/1852] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:34 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:35.073076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:35.093118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49116-eeb2-4fb5-ac22-f2867a35fb9a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:35.162440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:35.264140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:35.264740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.265049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.265374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.266018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:35.268291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:35.327543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 931/1853] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:35 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:35.331108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:35.354847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39ac8c-3825-4f82-805c-0ecb772b36e6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:35.367627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:35.369902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.369902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.369902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.369902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:35.383238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:35.416375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 923/1854] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:35.417719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:35.435823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a00d-16f0-4f86-bb84-6f993532a0db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:35.450689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:35.451354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.451644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.451914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.452571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:35.456909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afc76e-27d4-4b77-ae91-919abbd5eed8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:35.491325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 932/1855] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:35.496118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:35.513358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ebc7-1218-446f-bb8c-3789bb2065e8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:35.529008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:35.529008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.529008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.529008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] PATH_INFO: `/v3/projects/a296296ea2b9437aa1479286e588ecd5/users/bcfc139893014c748285b9447ed62259/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.568452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a296296ea2b9437aa1479286e588ecd5, user_id=bcfc139893014c748285b9447ed62259, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:35.571455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:35.573953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:35.599111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 924/1856] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:35 2022] PUT /identity/v3/projects/a296296ea2b9437aa1479286e588ecd5/users/bcfc139893014c748285b9447ed62259/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:35.611482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.611772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.612049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.706098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] MFA Rules not processed for user `bcfc139893014c748285b9447ed62259`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:35.707304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:35.816805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 933/1857] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:35.886510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:35.888767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:35.973883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72569986-b872-49a2-b142-6a5d8b58a7b5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:35.980791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:35.982131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:35.982601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:35.982975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:35.997070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:35.999092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49976944-6586-4dc0-93c6-556446c30dca service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:36.014333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 925/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:43.947629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:43.949095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:44.166670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25517c-219f-4d4e-bb49-6a0db6dffa2c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:44.175415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:44.176329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:44.178018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:44.178443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] PATH_INFO: `/v3/users/3d2341b48e424ccabfed2ecdd49b5fac` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:44.182451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2341b48e424ccabfed2ecdd49b5fac)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:44.186188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:44.283578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:44.294602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2341b48e424ccabfed2ecdd49b5fac'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:44.295160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2341b48e424ccabfed2ecdd49b5fac'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:44.304026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2341b48e424ccabfed2ecdd49b5fac'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:44.304860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2341b48e424ccabfed2ecdd49b5fac'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:44.332173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2341b48e424ccabfed2ecdd49b5fac'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:44.368070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 934/1859] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:43 2022] DELETE /identity/v3/users/3d2341b48e424ccabfed2ecdd49b5fac => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:44.371674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59eaf6-8f8e-4f65-a167-f7e325e25b83 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:44.372826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59eaf6-8f8e-4f65-a167-f7e325e25b83 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:44.373286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59eaf6-8f8e-4f65-a167-f7e325e25b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:44.423738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6c59eaf6-8f8e-4f65-a167-f7e325e25b83 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:44.427201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c59eaf6-8f8e-4f65-a167-f7e325e25b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:44.663347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 926/1860] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:44.694532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:44.694532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:44.866069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd6900-a325-4835-9a6e-ab927ed6c0e7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:44.870899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:44.872262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:44.872586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:44.872937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:44.901682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:44.906414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57306b-ddbe-4f0d-845b-33ec51fc3ab7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:44.933349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 935/1861] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:45.176783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:45.176783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:45.177310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:45.178618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 927/1862] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:45.204439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:45.205389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:45.205672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:45.241097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:45.242541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:45.456069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 936/1863] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:24:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:45.696775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:45.700574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:45.706465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:45.708511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:45.731883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ad1ea-234f-4b26-8020-e0364187b42d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:45.738541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:45.739312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:45.739680 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:45.740083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] PATH_INFO: `/v3/users/c88155549d79434fa91b72d8733cdb28` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:45.741604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88155549d79434fa91b72d8733cdb28)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:45.744257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:45.744395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90cfc0-a061-4282-89b0-86aa8e59c9b8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:45.761104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:45.762485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:45.763494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:45.763843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:45.776879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:45.778970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848d053-32f2-4f0c-bb5d-816bf6bba431 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:45.799429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 928/1864] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:45.829078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:45.836221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88155549d79434fa91b72d8733cdb28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:45.838545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88155549d79434fa91b72d8733cdb28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:45.858587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88155549d79434fa91b72d8733cdb28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:45.859866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88155549d79434fa91b72d8733cdb28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:45.877106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88155549d79434fa91b72d8733cdb28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:45.893974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 937/1865] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:45 2022] DELETE /identity/v3/users/c88155549d79434fa91b72d8733cdb28 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:45.898897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:45.899183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:45.899454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:45.919681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:45.920914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:45.989217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 929/1866] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:46.009636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:46.011068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:46.079006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3675c08-cda5-4edc-b0cf-762fd3068563 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:46.082686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:46.083296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:46.083569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:46.083841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:46.094564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:46.095566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:46.107714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 938/1867] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:46.503260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:46.508599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:46.571068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c33cce-0aed-4458-8faf-c99a5ebdce97 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:46.606181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:46.607235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:46.607774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:46.612151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] PATH_INFO: `/v3/projects/6570d1b5ea13405ea22570e38f080851` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:46.613630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6570d1b5ea13405ea22570e38f080851)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:46.630071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:46.642860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:46.651781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:46.697912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dee04-1382-4757-a683-e93cae5c7d46 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:46.709851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:46.710643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:46.710832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:46.711602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] PATH_INFO: `/v3/projects/2cfbae58e2d14e659c6e93cc39f64fe2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:46.718160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfbae58e2d14e659c6e93cc39f64fe2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:46.720261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:46.736698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:46.796535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6570d1b5ea13405ea22570e38f080851', 'request_id': 'req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.814737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6570d1b5ea13405ea22570e38f080851', 'request_id': 'req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.833303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfbae58e2d14e659c6e93cc39f64fe2', 'request_id': 'req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.837383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6570d1b5ea13405ea22570e38f080851', 'request_id': 'req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.844183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfbae58e2d14e659c6e93cc39f64fe2', 'request_id': 'req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.854895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfbae58e2d14e659c6e93cc39f64fe2', 'request_id': 'req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.862801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] The token cache is being invalidate because project 6570d1b5ea13405ea22570e38f080851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:46.863344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.864180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.865914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 930/1868] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:46 2022] DELETE /identity/v3/projects/6570d1b5ea13405ea22570e38f080851 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:46.870511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:46.873500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:46.894669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] The token cache is being invalidate because project 2cfbae58e2d14e659c6e93cc39f64fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:46.895030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.895966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:46.898563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 939/1869] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:46 2022] DELETE /identity/v3/projects/2cfbae58e2d14e659c6e93cc39f64fe2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:46.903958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:46.905795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.011564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:47.019149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb8b2-bd88-43a6-8bcb-ddafcd9cea82 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.058787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc72d95-c90c-4cd9-aa3e-26b82c70fc92 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.066658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.067314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.067647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.067847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] PATH_INFO: `/v3/users/5900f65df5cf4f00810d2217ff755b54` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.069203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5900f65df5cf4f00810d2217ff755b54)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.071160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.114618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.115368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.115718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.116109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] PATH_INFO: `/v3/users/a03bce60a10244a2a579118122e8cd87` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.117590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a03bce60a10244a2a579118122e8cd87)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.119882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.139420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5900f65df5cf4f00810d2217ff755b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.140168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5900f65df5cf4f00810d2217ff755b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.158756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5900f65df5cf4f00810d2217ff755b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.160003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5900f65df5cf4f00810d2217ff755b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.182484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5900f65df5cf4f00810d2217ff755b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.192702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a03bce60a10244a2a579118122e8cd87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.193465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a03bce60a10244a2a579118122e8cd87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.194450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a03bce60a10244a2a579118122e8cd87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.201145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 940/1870] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:46 2022] DELETE /identity/v3/users/5900f65df5cf4f00810d2217ff755b54 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:47.205956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a03bce60a10244a2a579118122e8cd87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.207402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:47.221874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a03bce60a10244a2a579118122e8cd87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.240752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 931/1871] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:46 2022] DELETE /identity/v3/users/a03bce60a10244a2a579118122e8cd87 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:47.499705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.501957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.546411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.548866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.618824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90045a6-dcca-439b-b976-868d2a42785f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.680020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fc0d1-b120-48b5-836e-6d6365b4a82b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.686529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.687184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.687453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.687728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] PATH_INFO: `/v3/projects/f75b61013ed44d75bc44951b0f7171b8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.695434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75b61013ed44d75bc44951b0f7171b8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.697459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.717812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.718451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.718723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.719024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] PATH_INFO: `/v3/projects/0b4e081954ad42c985b2b88a966f6030` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.720464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4e081954ad42c985b2b88a966f6030)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.722388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.740463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:47.751195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75b61013ed44d75bc44951b0f7171b8', 'request_id': 'req-40a76ca6-21f7-4839-bc96-224070271fe7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.757844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75b61013ed44d75bc44951b0f7171b8', 'request_id': 'req-40a76ca6-21f7-4839-bc96-224070271fe7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.766742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75b61013ed44d75bc44951b0f7171b8', 'request_id': 'req-40a76ca6-21f7-4839-bc96-224070271fe7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.774464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4e081954ad42c985b2b88a966f6030', 'request_id': 'req-0b50f030-ea76-4416-be72-2a698d745880'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.778739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] The token cache is being invalidate because project f75b61013ed44d75bc44951b0f7171b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:47.779027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.779715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a76ca6-21f7-4839-bc96-224070271fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.780358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4e081954ad42c985b2b88a966f6030', 'request_id': 'req-0b50f030-ea76-4416-be72-2a698d745880'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.781225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 932/1872] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:47 2022] DELETE /identity/v3/projects/f75b61013ed44d75bc44951b0f7171b8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:47.786173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2173-a929-48a8-a63d-8a143b8a85d6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.786447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2173-a929-48a8-a63d-8a143b8a85d6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.786733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2173-a929-48a8-a63d-8a143b8a85d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.788721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4e081954ad42c985b2b88a966f6030', 'request_id': 'req-0b50f030-ea76-4416-be72-2a698d745880'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.802003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] The token cache is being invalidate because project 0b4e081954ad42c985b2b88a966f6030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:47.802654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.802929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0b50f030-ea76-4416-be72-2a698d745880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:47.804409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 941/1873] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:47 2022] DELETE /identity/v3/projects/0b4e081954ad42c985b2b88a966f6030 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:47.808285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bc4c2173-a929-48a8-a63d-8a143b8a85d6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:47.810438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db67e-7fae-4862-b13a-a3e71c62d349 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.810557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bc4c2173-a929-48a8-a63d-8a143b8a85d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.810717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db67e-7fae-4862-b13a-a3e71c62d349 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.810988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db67e-7fae-4862-b13a-a3e71c62d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.830386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-458db67e-7fae-4862-b13a-a3e71c62d349 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:47.831571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-458db67e-7fae-4862-b13a-a3e71c62d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.877741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 933/1874] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:47.890371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.890803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.891135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.897666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 942/1875] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:47.899611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:47.907186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.907950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.907950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.916091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:47.917441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.926616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 934/1876] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:47.929800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:47.930845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.931257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:47.939602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 943/1877] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:47.940116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a93f-c508-4d1f-aeb8-e8d1f4328b98 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.943682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.944728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.945352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.945638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.945899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.946568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:47.946865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.948442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.952825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72b508-78e4-4387-a4f3-417847affde2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.958211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.958812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.959087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.959353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.959996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:47.960296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.962030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.968037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 935/1878] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:47.969752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.978325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 944/1879] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:47.979307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:47.982059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86de09-1278-4dab-a450-64f89d94249a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.986670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:47.987278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:47.987551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:47.987817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:47.988446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:47.990425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:47.991875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5db-0e78-4419-a05d-d20ec37e36a0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:47.999106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.000034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.000384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.000726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.001909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.004623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.036984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 936/1880] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.038313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.039072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 945/1881] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:47 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.043390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.118297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afed3-e99a-468b-bfb1-f717239c8e95 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.120466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae1d0-dc9a-4065-898e-8c635e6c2da7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.125003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.125830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.126078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.126774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.126774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.126976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.127447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.127598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.127954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.128533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.129750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.130463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.140987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:48.160098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 946/1882] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.162232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.177708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d24a09-c3fc-44a5-a593-853ac5f11e02 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.185766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 937/1883] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:48.186686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.189896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.191220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.192777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.193065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.193694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.203713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.213805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b34f-4714-4171-91ef-926af1642ae9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.219538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.219955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 947/1884] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:48.220524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.220964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.221385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.222168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.223864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.224118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.234455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72941fb-c4bf-441c-9996-c743004f3c40 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.238526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 938/1885] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:48.239551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.241898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.242503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.242778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.243045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.243684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.245326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.252979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e293-2bd7-490f-81ee-e4c9504f0889 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.258928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 948/1886] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:48.263393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.271674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.272284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.272558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.272828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.272918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2530c-4e07-4643-a148-8a8058fdf65a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.273508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.275114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.277988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.278708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.279056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.279396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] PATH_INFO: `/v3/projects/731cff11a7be4c72b87a451ed310324c/users/6c85169511a54e46b1825025ebac5106/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.286175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 939/1887] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:48.290306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.294960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] RBAC: Authorizing `identity:create_grant(project_id=731cff11a7be4c72b87a451ed310324c, user_id=6c85169511a54e46b1825025ebac5106, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.297739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45becc08-1ca4-4499-aa78-71c728ab959d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.301070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b363de-b3ec-4144-9bbc-21faa6513fd0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.306421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.307121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.307419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.307757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] PATH_INFO: `/v3/projects/b0e7e9eadb5f4ecb91a3e72237d01287/users/e38801e6e7414c069e64902c2d612463/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.313059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 949/1888] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:48 2022] PUT /identity/v3/projects/731cff11a7be4c72b87a451ed310324c/users/6c85169511a54e46b1825025ebac5106/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:48.323529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.323808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.324081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.326371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e7e9eadb5f4ecb91a3e72237d01287, user_id=e38801e6e7414c069e64902c2d612463, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.329071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f54be5-091c-498a-b9a3-b6c3407f1b52 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.344115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 940/1889] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:48 2022] PUT /identity/v3/projects/b0e7e9eadb5f4ecb91a3e72237d01287/users/e38801e6e7414c069e64902c2d612463/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:48.352719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] MFA Rules not processed for user `6c85169511a54e46b1825025ebac5106`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:48.353947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:48.354721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.355011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.355285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.387501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] MFA Rules not processed for user `e38801e6e7414c069e64902c2d612463`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:48.388732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:48.424044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 950/1890] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:48.428797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.446345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:48.456628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 941/1891] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:48.461092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.499799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c675e3-4ed8-4871-99d8-b94dd29176bd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.504381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.505034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.505339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.505608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.506272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.508175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.525549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305c43-7ff8-441f-a26d-038ee852b79c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.531548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.532175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.532451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.532721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.533406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.535306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.540902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 951/1892] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.541954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.567937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:48.568985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 942/1893] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:48.572632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.614354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e76cc-5f3b-4d8a-823a-5dac4f0ac524 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.634039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c7087-6c39-49d6-92ba-2759c10928e4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.638820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.639361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.639638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.639908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.640575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.642510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.663729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 943/1894] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.667424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.675144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.675826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.676036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.676310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.676932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.678325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f108-8abf-47fa-a4a9-8225de7c64cc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.679188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.683553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.684422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.684823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.685135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.685779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.687349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.701332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 944/1895] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:48.706022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.712208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 952/1896] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:24:48.716772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.718545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed44f-1829-40ce-97b5-0d1ef8eae10c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.724707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.725353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.725628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.726652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.727310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.727599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4424f-610f-4ba7-86e2-18bf114542f2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.729616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.733241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.733992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.734386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.734751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.735493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.739074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.745551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 945/1897] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:48.746550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.748702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 953/1898] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:48.752538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.757035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fbb7c-01d4-4bdf-b60d-1e7e13f6fb9b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.763489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.763781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17128-41b0-4c76-8f09-9d3c7f116f66 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.764153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.764428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.764805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] PATH_INFO: `/v3/projects/d465f42e9b52454fb0c97f9660d03e40/users/dc310581115742f5bd1a0c3974ecaeb3/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.772335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.773045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.773325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.773703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] PATH_INFO: `/v3/projects/381c36a8faa9428dabe4c2847d86f279/users/47807ddeb5b1495c99333ff5b9ba90f6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.785538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d465f42e9b52454fb0c97f9660d03e40, user_id=dc310581115742f5bd1a0c3974ecaeb3, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.788230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d28101-dd1f-4f24-ae7f-feb1d358364f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.792349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381c36a8faa9428dabe4c2847d86f279, user_id=47807ddeb5b1495c99333ff5b9ba90f6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.795987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.804011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 946/1899] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:48 2022] PUT /identity/v3/projects/d465f42e9b52454fb0c97f9660d03e40/users/dc310581115742f5bd1a0c3974ecaeb3/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:48.815632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.815985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.816500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.821986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 954/1900] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:48 2022] PUT /identity/v3/projects/381c36a8faa9428dabe4c2847d86f279/users/47807ddeb5b1495c99333ff5b9ba90f6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:48.826023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.848671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] MFA Rules not processed for user `dc310581115742f5bd1a0c3974ecaeb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:48.849914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:48.862981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:48.899184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683712c-b9e4-4fe8-892b-1aaf8dc52ab6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.905053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.905662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.905943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.906215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.906857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.908729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.920630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 955/1901] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:48.920965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 947/1902] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:48.924937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.926018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:48.936546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0899-1ef3-4504-8389-fc9d66270034 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.937530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73983e14-0809-47a1-b903-e32b8e7b5135 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:48.941596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.942304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:48.942414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.942683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.942897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.943090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] PATH_INFO: `/v3/projects/381c36a8faa9428dabe4c2847d86f279/users/47807ddeb5b1495c99333ff5b9ba90f6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.943228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.943463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.944097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.945343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=381c36a8faa9428dabe4c2847d86f279, user_id=47807ddeb5b1495c99333ff5b9ba90f6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:48.946093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.947850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997995c-c247-4a52-a9f0-a641ae3d4d2c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:48.958923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 956/1903] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:48 2022] PUT /identity/v3/projects/381c36a8faa9428dabe4c2847d86f279/users/47807ddeb5b1495c99333ff5b9ba90f6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:48.970201 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:48.970572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:48.970925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:48.976787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 948/1904] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:48.980107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:48.981340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.002439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] MFA Rules not processed for user `47807ddeb5b1495c99333ff5b9ba90f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:49.003649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.044768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908e164-112e-462d-bc29-efd3fc820562 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.051435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.052045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.052319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.052590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.053267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.055181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.079979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 949/1905] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:49.081320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 957/1906] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:49.083796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.094437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.095499 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f9a0a-e736-4b69-93c5-aaebbd79bed5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.095860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.100560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.101487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.101767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.102060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.102679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.104257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.115369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 950/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:49.119617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.138244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ef89-4a02-4daf-b267-376fe07af2c1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.143476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.144071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.144344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.144742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] PATH_INFO: `/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35/users/75391c2757574bea8e9704b3398978c6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.158316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd1de1-4276-4c1d-821a-7803b769a13e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.161872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b90d343c60848e1bab2f9d16b5e8a35, user_id=75391c2757574bea8e9704b3398978c6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.162174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.162850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.163196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.163552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.164710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.173984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.175410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.178082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 951/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:49 2022] PUT /identity/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35/users/75391c2757574bea8e9704b3398978c6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:49.182158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.185619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 958/1909] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:24:49.217879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.228698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7346260-d0e7-494d-97d6-e01b4ea46ccd service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.232838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:49.251617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef47d2a-7b2c-4585-bcde-4a04c2a6a954 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.260121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.260806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.261172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.261503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.262842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.264656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.276797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 952/1910] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:49.282013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.299054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaddd-334e-4556-9f9f-cb1edb8cd4a6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.308268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.310625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.312222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.314071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] PATH_INFO: `/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35/users/75391c2757574bea8e9704b3398978c6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.314235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.314896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.315156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.315429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] PATH_INFO: `/v3/users/6c85169511a54e46b1825025ebac5106` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.316667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c85169511a54e46b1825025ebac5106)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.318762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.323732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b90d343c60848e1bab2f9d16b5e8a35, user_id=75391c2757574bea8e9704b3398978c6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.328428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cb725-d5e8-4e7c-b260-20dd10312d84 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.349005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 953/1911] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:49 2022] PUT /identity/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35/users/75391c2757574bea8e9704b3398978c6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:49.351458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.351988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.352260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.366082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:49.382528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c85169511a54e46b1825025ebac5106'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:49.383585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c85169511a54e46b1825025ebac5106'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:49.389254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] MFA Rules not processed for user `75391c2757574bea8e9704b3398978c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:49.391020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.396427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c85169511a54e46b1825025ebac5106'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:49.397232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c85169511a54e46b1825025ebac5106'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:49.426427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f5fe6e92-171d-4aa7-9c1d-d74d755f71ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c85169511a54e46b1825025ebac5106'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:49.443891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 959/1912] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:49 2022] DELETE /identity/v3/users/6c85169511a54e46b1825025ebac5106 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:49.448532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.448823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.449197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.470547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:49.471768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.475913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 954/1913] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:49.490883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.492714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.554245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 960/1914] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:49.559248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6702d-7db4-47ab-8830-0bc6d69b8137 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.565101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.566964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.613867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.614476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.614753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.615026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.627799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff62f-5eb7-4c37-b143-6b562a3fa417 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.631411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.632098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.632443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.632846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.641866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:49.645270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.646631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.658492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 961/1915] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:49.682196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.684147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.695341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 955/1916] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:49.739709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.741254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.753879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1a4f-2cde-4d6e-be3a-1a7d24311e96 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.757511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.758111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.758390 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.758659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.760396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.831386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:49.835588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:49.836927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:49.849177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 962/1917] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:49.975777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:49.977236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:49.991460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81089-3c2e-4edc-869d-64d4324f10d5 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:49.996748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:49.997306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:49.997584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:49.997856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] PATH_INFO: `/v3/users/bcfc139893014c748285b9447ed62259` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:49.998991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfc139893014c748285b9447ed62259)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:50.000912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:50.019659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:50.022138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.039419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122664-eac4-4196-900f-a0a250d2278c service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:50.056377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:50.057063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:50.057347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:50.057619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] PATH_INFO: `/v3/projects/731cff11a7be4c72b87a451ed310324c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:50.060460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=731cff11a7be4c72b87a451ed310324c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:50.062769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:50.067967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfc139893014c748285b9447ed62259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.068802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfc139893014c748285b9447ed62259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.069752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfc139893014c748285b9447ed62259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.079166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfc139893014c748285b9447ed62259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.095923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e1d7e604-6e2f-4702-a25f-5fbc25aebea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfc139893014c748285b9447ed62259'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.116078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 956/1918] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:49 2022] DELETE /identity/v3/users/bcfc139893014c748285b9447ed62259 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:50.120572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:50.120859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:50.122130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:50.142118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:50.143311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.149862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731cff11a7be4c72b87a451ed310324c', 'request_id': 'req-d9818a7a-d9da-4311-9372-7958fa32ad7a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.167074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731cff11a7be4c72b87a451ed310324c', 'request_id': 'req-d9818a7a-d9da-4311-9372-7958fa32ad7a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.179555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731cff11a7be4c72b87a451ed310324c', 'request_id': 'req-d9818a7a-d9da-4311-9372-7958fa32ad7a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.198177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] The token cache is being invalidate because project 731cff11a7be4c72b87a451ed310324c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:50.198576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.205399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.207364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 963/1919] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:50 2022] DELETE /identity/v3/projects/731cff11a7be4c72b87a451ed310324c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:50.210978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:50.212408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.295560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 957/1920] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:50.305550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:50.306694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.308342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818a7a-d9da-4311-9372-7958fa32ad7a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:50.316655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:50.317490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:50.317898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:50.318439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] PATH_INFO: `/v3/users/47807ddeb5b1495c99333ff5b9ba90f6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:50.320017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47807ddeb5b1495c99333ff5b9ba90f6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:50.323749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:50.390534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:50.417712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47807ddeb5b1495c99333ff5b9ba90f6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.419304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47807ddeb5b1495c99333ff5b9ba90f6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.420925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc75f3-0859-4c11-bff5-6ef56fef3b5b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:50.427522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:50.428391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:50.428800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47807ddeb5b1495c99333ff5b9ba90f6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.429186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:50.429794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:50.430649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47807ddeb5b1495c99333ff5b9ba90f6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.432760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.445727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1695397b-184d-4a1b-b252-ce414e7d6979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47807ddeb5b1495c99333ff5b9ba90f6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:50.451262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:50.451747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:50.453160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:50.466328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 958/1921] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:24:50.467714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 964/1922] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:50 2022] DELETE /identity/v3/users/47807ddeb5b1495c99333ff5b9ba90f6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:50.900486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:50.902888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:50.930214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252482ec-0d92-43e3-a4ab-bd2ff7438f61 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.016179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.016937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.017285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.017560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] PATH_INFO: `/v3/projects/a296296ea2b9437aa1479286e588ecd5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.020004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a296296ea2b9437aa1479286e588ecd5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.022197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.075251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a296296ea2b9437aa1479286e588ecd5', 'request_id': 'req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:51.081532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a296296ea2b9437aa1479286e588ecd5', 'request_id': 'req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:51.091144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a296296ea2b9437aa1479286e588ecd5', 'request_id': 'req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:51.103838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] The token cache is being invalidate because project a296296ea2b9437aa1479286e588ecd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:51.104274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:51.105071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0eddcd5-9b96-46bc-aaba-8ee12ae9230e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:51.106757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 959/1923] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:50 2022] DELETE /identity/v3/projects/a296296ea2b9437aa1479286e588ecd5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:51.112372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.112891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.112891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.133849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:51.135062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:51.210232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 965/1924] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:51.212267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.212552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.215979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.239677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:51.241580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:51.251208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 960/1925] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:51.255303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.267968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacca3b-870e-4d07-aca5-316338399dd7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.273121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.273815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.274567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.274567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.275202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:51.275572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.277357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.289970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 966/1926] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:51.295206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.306413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1eb1e0-5a8e-4a8f-bbc3-783e6f9d9782 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.312604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.313355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.313709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.314048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.314757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.316727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.388599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 961/1927] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:51.392554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.406767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4e79e-0c39-4398-a23b-bbd1f0d0f319 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.572104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.573018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.573440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.573899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.575140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.577608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.619345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 967/1928] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:51.620350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.631827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e87b5-e681-4a0b-9956-7a161e5b792b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.637303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.638726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.639219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.639637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.640497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.642302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.666465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.673227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 962/1929] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:51.679692 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6bf3-cf19-4811-aa8c-e82659456753 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.689909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.690588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.690871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.691147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.692121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.693493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d26d71-6e4c-4475-a741-715870720d5f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.716065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.717339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 968/1930] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:51.733086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92d1e1-7590-4aea-8e92-ee4854edc1c5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:51.738803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:51.739556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.739741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.740024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] PATH_INFO: `/v3/projects/3104f093476346d4889c945c362a6a8e/users/9e2de5c5b5334744a739f7e1ab7d11c3/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.757218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3104f093476346d4889c945c362a6a8e, user_id=9e2de5c5b5334744a739f7e1ab7d11c3, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:51.760409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:51.781264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 963/1931] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:51 2022] PUT /identity/v3/projects/3104f093476346d4889c945c362a6a8e/users/9e2de5c5b5334744a739f7e1ab7d11c3/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:51.793115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:51.793541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:51.793706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:51.829705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] MFA Rules not processed for user `9e2de5c5b5334744a739f7e1ab7d11c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:51.831029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:51.903011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:51.926217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 969/1932] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:51.930690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:51.935911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:52.023547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494848-f650-4f33-adbc-b6dce82cda2a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.033382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.033382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.033664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.034515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.035816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.039990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.041800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:52.071257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.101840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7be42-818d-4d48-856b-8c9e90d12c7d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.110828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.111478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.113827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.114372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] PATH_INFO: `/v3/projects/381c36a8faa9428dabe4c2847d86f279` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.119096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=381c36a8faa9428dabe4c2847d86f279)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.121216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.146172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 964/1933] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:51 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:52.147709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.235665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '381c36a8faa9428dabe4c2847d86f279', 'request_id': 'req-05695935-f818-4c6a-b34f-1101eda7d765'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:52.250245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '381c36a8faa9428dabe4c2847d86f279', 'request_id': 'req-05695935-f818-4c6a-b34f-1101eda7d765'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:52.257695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371adcd7-39bd-4fc7-a371-16ca9da264a2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.262358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '381c36a8faa9428dabe4c2847d86f279', 'request_id': 'req-05695935-f818-4c6a-b34f-1101eda7d765'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:52.285733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] The token cache is being invalidate because project 381c36a8faa9428dabe4c2847d86f279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:52.286023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:52.287538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-05695935-f818-4c6a-b34f-1101eda7d765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:52.292744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 970/1934] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:52 2022] DELETE /identity/v3/projects/381c36a8faa9428dabe4c2847d86f279 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:52.297380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ecaf2-eb27-4cdd-9856-0252e433515e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.297670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ecaf2-eb27-4cdd-9856-0252e433515e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.297957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ecaf2-eb27-4cdd-9856-0252e433515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.344143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c24ecaf2-eb27-4cdd-9856-0252e433515e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:52.345783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c24ecaf2-eb27-4cdd-9856-0252e433515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:52.482916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 971/1935] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:52.491433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.491705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.491981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.500427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.501030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.501297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.503903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.509034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.521237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.550334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:52.552147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:52.586511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 965/1936] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/users => generated 293 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:52.590278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 972/1937] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:52.594087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.595861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.595993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:52.608393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4d9a8-aac4-4ba9-a857-57c6ce6f34bb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.615076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.615747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.616027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.617026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.617757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:52.618065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.619034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30f4d-842a-4048-9948-0c21486768e2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.619723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.646008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 973/1938] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:52.650579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.650840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.651744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.651986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.652779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.654262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.656093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.669930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 966/1939] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:52.671004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4850-f71c-4c57-a55d-d9a0d579e6e1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.673864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.678119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.678951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.680040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.681071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.681773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.685951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.690161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdf5d-1a58-4339-97bd-702210c3f9fc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.697243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.698199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.698653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.699913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.708176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.710932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.740543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 967/1940] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:52.749891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 974/1941] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:52.750052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.751439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.752890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:52.762278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9041a-6b36-4cd6-abb9-0d0ae6e3f04e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.815042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:52.852897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.853793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.854087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.854400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] PATH_INFO: `/v3/projects/c30e6035ff924cee80f8b865fe6d9f99/users/994427ebbf5e44838a63dcc9facec834/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.865682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef23f0-265a-417c-b455-a463e820cfd8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.871248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.872046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.872393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.872726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.873626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.875598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.892767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30e6035ff924cee80f8b865fe6d9f99, user_id=994427ebbf5e44838a63dcc9facec834, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.897612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce410da-2480-44d8-8565-a40d822110fa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:52.900294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 975/1942] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:52.905152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:52.915771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 968/1943] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:52 2022] PUT /identity/v3/projects/c30e6035ff924cee80f8b865fe6d9f99/users/994427ebbf5e44838a63dcc9facec834/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:52.916035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76459-8756-490f-a769-d640dd4386ad admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:52.926924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.927697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.928169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.962781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] MFA Rules not processed for user `994427ebbf5e44838a63dcc9facec834`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:52.964263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:52.990274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:52.991068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:52.991209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:52.991411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:52.992192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:52.993944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.010568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 976/1944] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:53.015139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.025334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf9f4-5f23-46ea-82bc-4b03c41650b9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.030781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.031383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.031652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.032552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.032760 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.034816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.046023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 969/1945] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:53.046607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 977/1946] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:53.050143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.052559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.062481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0cba2-460f-40b2-bfce-1b8b1e5da8d1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.067678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.068345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.068624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.068914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] PATH_INFO: `/v3/projects/16ec42b66c0c4c85bbaba6d624b309aa/users/a47410e2865a4243b4a1b2cef8325643/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.070211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22299589-8235-4213-9c9a-cc506e93ea71 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.075161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.075770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.076045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.076312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.077002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.079081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.092770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ec42b66c0c4c85bbaba6d624b309aa, user_id=a47410e2865a4243b4a1b2cef8325643, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.096350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bb873-aac3-4b69-8401-bdfdde98bac3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.116065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 970/1947] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:53.116543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 978/1948] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:53 2022] PUT /identity/v3/projects/16ec42b66c0c4c85bbaba6d624b309aa/users/a47410e2865a4243b4a1b2cef8325643/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:53.126204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.128146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.128146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.128358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.166011 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] MFA Rules not processed for user `a47410e2865a4243b4a1b2cef8325643`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:53.167634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.219289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:53.231881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247912dc-1f2f-4dbb-9bc5-08c7fa78260a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.237012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.237722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.237975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.238251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.238893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.240805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.259809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 979/1949] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:53.261021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.264419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.269412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 971/1950] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:53.269412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.281078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f2d1b-d9ec-492e-8c36-ec5645265bbd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.281375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04174-8c47-47a8-9c8b-17b0b97c82e9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.286266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.286839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.287005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.287390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.287493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.287665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.287736 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.288094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] PATH_INFO: `/v3/users/e38801e6e7414c069e64902c2d612463` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.288448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.290456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.295054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38801e6e7414c069e64902c2d612463)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.298009 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.319229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:53.326823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 972/1951] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:53.327913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.381830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38801e6e7414c069e64902c2d612463'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:53.383573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38801e6e7414c069e64902c2d612463'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:53.394333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38801e6e7414c069e64902c2d612463'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:53.395318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38801e6e7414c069e64902c2d612463'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:53.412358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38801e6e7414c069e64902c2d612463'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:53.441836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 980/1952] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:53 2022] DELETE /identity/v3/users/e38801e6e7414c069e64902c2d612463 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:53.442698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.447528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.450659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726c8a-18f3-4f67-b8b7-dc54afe8417f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.561492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.562219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.562812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.563096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.563766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.565982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d329dc-3a61-4095-bc17-1154d87a39d1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.595003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 973/1953] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:53.604199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.604716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.606142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.609656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd6f03-bbc7-47ca-aa59-51879f5f3e9e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.624915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.625568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.625843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.626112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.626761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.628912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.635672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:53.637762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.656804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 974/1954] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:53.656804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.657576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.685206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1c360-9e65-41ee-b28a-4bac113816f0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.686531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 981/1955] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:53 2022] POST /identity/v3/users => generated 303 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:53.687502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.698755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.701422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.702368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.703627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] PATH_INFO: `/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e/users/b2102718fbef4f499768ea6f81f2c75d/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.710981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:53.750326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec09e7698f54d97a7b57259d4e4f39e, user_id=b2102718fbef4f499768ea6f81f2c75d, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.753256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.787004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 975/1956] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:53 2022] PUT /identity/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e/users/b2102718fbef4f499768ea6f81f2c75d/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:53.791728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.868918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1eebb-fd9a-4e95-8102-43dcfed20b1b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.872517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:53.915425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b2bab-1f44-4fcf-a675-06d91f4f1bb2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.924703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.925874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:53.926263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:53.926761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:53.927682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:53.929629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:53.943545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:53.946341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 976/1957] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:53.948141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:53.987133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f41c91-d933-4714-bcb8-4a84b06014cd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:53.999266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:53.999898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.000176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.000455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.001141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.002126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.002442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.002752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.003297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.006697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.018542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.020471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.042568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 982/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:54.044272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.057037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab8f9-2f55-4840-ac09-05e712438c95 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.061867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 977/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:54.064206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.065555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.066026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.066454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.066970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] PATH_INFO: `/v3/projects/6a085ca6fd6645b281410787093d06dd/users/e0a1ed1b2a33498d9222f2de20bbedc2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.088025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4b4f5-d267-4a68-a3b2-37307e9e91ab admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.091126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a085ca6fd6645b281410787093d06dd, user_id=e0a1ed1b2a33498d9222f2de20bbedc2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.097404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.104338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.105270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.106150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.106691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] PATH_INFO: `/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e/users/b2102718fbef4f499768ea6f81f2c75d/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.111584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec09e7698f54d97a7b57259d4e4f39e, user_id=b2102718fbef4f499768ea6f81f2c75d, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.118816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55937-2907-41c4-b2a2-0e043f02ce57 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.123723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 983/1960] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:54 2022] PUT /identity/v3/projects/6a085ca6fd6645b281410787093d06dd/users/e0a1ed1b2a33498d9222f2de20bbedc2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:54.124765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.145494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 978/1961] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:54 2022] PUT /identity/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e/users/b2102718fbef4f499768ea6f81f2c75d/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:54.172431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b1eec-8787-427d-8f91-63a89724caf2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.173295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b1eec-8787-427d-8f91-63a89724caf2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.173580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b1eec-8787-427d-8f91-63a89724caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.245939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d177d6-01e9-4f7f-bac9-f517b1bb182d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.254467 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-114b1eec-8787-427d-8f91-63a89724caf2 None None] MFA Rules not processed for user `b2102718fbef4f499768ea6f81f2c75d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:54.257931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-114b1eec-8787-427d-8f91-63a89724caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.435726 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 979/1962] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:54.437651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.438567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.439388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.444050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 980/1963] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:24:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:24:54.444884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.450811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.450811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.450811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.451150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.451622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.453242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.454530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.483961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 984/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:54.486229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.498498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9061ab0-a72c-436d-b516-aacafa6f59ec None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.499805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.514860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.515509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.516287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.516570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] PATH_INFO: `/v3/users/8d413510d5344e9b85b44ec40bd232a7` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.520792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d413510d5344e9b85b44ec40bd232a7)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.522542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.628876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d413510d5344e9b85b44ec40bd232a7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:54.630385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d413510d5344e9b85b44ec40bd232a7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:54.632894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d413510d5344e9b85b44ec40bd232a7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:54.648387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d413510d5344e9b85b44ec40bd232a7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:54.669337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1f4e-cebe-43bc-a85d-64221700a5b2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.674999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.675691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.676033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.676364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.679429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.692530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d413510d5344e9b85b44ec40bd232a7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:54.697030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:54.711191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 981/1965] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:54 2022] DELETE /identity/v3/users/8d413510d5344e9b85b44ec40bd232a7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:54.712046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.713750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.727025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:54.776459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.778582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7981c9f-b12e-4a11-8a5c-bc4a795bf2eb service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.788686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 985/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:54.790442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9cc03-fbbe-4ddd-b759-2b9e5b2939c3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.790795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9cc03-fbbe-4ddd-b759-2b9e5b2939c3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.791135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9cc03-fbbe-4ddd-b759-2b9e5b2939c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.812752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b1891-e8b0-4461-a379-3be821849473 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:54.819919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:54.820853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.821422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.821704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] PATH_INFO: `/v3/projects/6a085ca6fd6645b281410787093d06dd/users/e0a1ed1b2a33498d9222f2de20bbedc2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.825992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a085ca6fd6645b281410787093d06dd, user_id=e0a1ed1b2a33498d9222f2de20bbedc2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:54.829093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79100c-68db-47b5-92b6-2addafd3b7c4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:54.841161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0af9cc03-fbbe-4ddd-b759-2b9e5b2939c3 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:54.842383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0af9cc03-fbbe-4ddd-b759-2b9e5b2939c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.845191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 982/1967] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:54 2022] PUT /identity/v3/projects/6a085ca6fd6645b281410787093d06dd/users/e0a1ed1b2a33498d9222f2de20bbedc2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:54.846274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.846654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.846931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.873558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:54.874933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:54.966296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 986/1968] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:54.967028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:54.967321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:54.967596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:54.972287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 983/1969] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:54.984248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:54.988110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:55.054649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] MFA Rules not processed for user `e0a1ed1b2a33498d9222f2de20bbedc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:55.056185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:55.101508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc5650-b91f-4d67-8d39-bd87f1daf3f0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:55.113234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:55.113862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:55.114136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:55.114399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:55.141793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:55.143479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:55.158659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 984/1970] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:55.159567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:55.161235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:55.171338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 987/1971] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:55.183064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119f4d1-c9e8-486d-bc3c-46c712cafcd2 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:55.185554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:55.187241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:55.190394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:55.191508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:55.191790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:55.192061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:55.203433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:55.206033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:55.232616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 985/1972] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:55.255431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:55.338349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f425da-a2fd-4618-8524-c74e63b6f3d1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:55.343073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:55.343672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:55.344059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:55.344420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:55.359813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:55.361307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:55.392457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 988/1973] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:55.483448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:55.503682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b8834-6349-4a82-8b06-e2ec092ad55f service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:55.509345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:55.510087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:55.510473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:55.510844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:55.514001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:55.567243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:55.628348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:55.644630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:55.681386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 986/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:55.972473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:55.974401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:56.012628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc675-ac54-47e6-a625-e44a2fd4bf75 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:56.025215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:56.026520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:56.027025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:56.028483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] PATH_INFO: `/v3/projects/b0e7e9eadb5f4ecb91a3e72237d01287` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:56.030508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e7e9eadb5f4ecb91a3e72237d01287)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:56.039615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:56.040362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:56.041459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:56.071957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac4b30-1416-4d19-8421-66fe97cdcbce service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:56.087985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:56.088984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:56.089131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:56.089487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] PATH_INFO: `/v3/projects/de797be1153940df900f635163c09d94` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:56.092131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de797be1153940df900f635163c09d94)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:56.098758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:56.180178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e7e9eadb5f4ecb91a3e72237d01287', 'request_id': 'req-9379efda-270e-4a0b-8fde-05f58a22afa8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.196574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e7e9eadb5f4ecb91a3e72237d01287', 'request_id': 'req-9379efda-270e-4a0b-8fde-05f58a22afa8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.200513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de797be1153940df900f635163c09d94', 'request_id': 'req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.213651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e7e9eadb5f4ecb91a3e72237d01287', 'request_id': 'req-9379efda-270e-4a0b-8fde-05f58a22afa8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.218536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de797be1153940df900f635163c09d94', 'request_id': 'req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.235385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] The token cache is being invalidate because project b0e7e9eadb5f4ecb91a3e72237d01287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:56.235619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.236369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.238033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de797be1153940df900f635163c09d94', 'request_id': 'req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.248372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 989/1975] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:55 2022] DELETE /identity/v3/projects/b0e7e9eadb5f4ecb91a3e72237d01287 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:56.250448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:56.254081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] The token cache is being invalidate because project de797be1153940df900f635163c09d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:56.254494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.255530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:56.256940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.259324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 987/1976] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:56 2022] DELETE /identity/v3/projects/de797be1153940df900f635163c09d94 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:56.264246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:56.267883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:56.341492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379efda-270e-4a0b-8fde-05f58a22afa8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:56.359311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b948e7-cba1-47e4-a085-2b6ea57ba02f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:56.367044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:56.368083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:56.368474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:56.368826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] PATH_INFO: `/v3/users/2a789ef1ae0d47c3bf85da2a668d7a8a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:56.370291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a789ef1ae0d47c3bf85da2a668d7a8a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:56.374346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:56.403200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:56.411401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:56.412228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:56.412586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:56.413014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] PATH_INFO: `/v3/users/dc310581115742f5bd1a0c3974ecaeb3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:56.414454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc310581115742f5bd1a0c3974ecaeb3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:56.419566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:56.433675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a789ef1ae0d47c3bf85da2a668d7a8a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.434552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a789ef1ae0d47c3bf85da2a668d7a8a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.435182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a789ef1ae0d47c3bf85da2a668d7a8a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.437581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:56.445759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a789ef1ae0d47c3bf85da2a668d7a8a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.463366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a789ef1ae0d47c3bf85da2a668d7a8a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.479307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 988/1977] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:56 2022] DELETE /identity/v3/users/2a789ef1ae0d47c3bf85da2a668d7a8a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:56.483723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc310581115742f5bd1a0c3974ecaeb3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.483723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc310581115742f5bd1a0c3974ecaeb3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.490519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc310581115742f5bd1a0c3974ecaeb3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.491261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc310581115742f5bd1a0c3974ecaeb3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.507467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc310581115742f5bd1a0c3974ecaeb3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:56.525772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 990/1978] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:56 2022] DELETE /identity/v3/users/dc310581115742f5bd1a0c3974ecaeb3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:56.993160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:56.996820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.076420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e2929-a344-41a1-8b46-38065ce4bf55 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.136113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.138196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.142983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.143989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.145415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.145928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.149348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.257348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.258886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.272464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c17d-8a47-40c6-8802-9c9e51e460ab admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.273812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 989/1979] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:57.279902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.280700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.281195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.281830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] PATH_INFO: `/v3/projects/d465f42e9b52454fb0c97f9660d03e40` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.283118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d465f42e9b52454fb0c97f9660d03e40)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.285448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.328667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.330609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.354000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc09be2-7ceb-45aa-8ede-1feb08b9c5b3 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.368799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.370024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.370488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.371671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] PATH_INFO: `/v3/projects/87e693b4ca95412db65788d16efc04c8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.373307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e693b4ca95412db65788d16efc04c8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.375886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.388350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd465f42e9b52454fb0c97f9660d03e40', 'request_id': 'req-def146d6-91b8-4bae-8143-7e18cada0703'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.415181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd465f42e9b52454fb0c97f9660d03e40', 'request_id': 'req-def146d6-91b8-4bae-8143-7e18cada0703'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.444100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd465f42e9b52454fb0c97f9660d03e40', 'request_id': 'req-def146d6-91b8-4bae-8143-7e18cada0703'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.462867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:57.466719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] The token cache is being invalidate because project d465f42e9b52454fb0c97f9660d03e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:57.467601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.470005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.477557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 991/1980] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:57 2022] DELETE /identity/v3/projects/d465f42e9b52454fb0c97f9660d03e40 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:57.480837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.485712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.490525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e693b4ca95412db65788d16efc04c8', 'request_id': 'req-bec0c362-09c5-421e-8ff7-d0947adc9a87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.500247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e693b4ca95412db65788d16efc04c8', 'request_id': 'req-bec0c362-09c5-421e-8ff7-d0947adc9a87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.514450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e693b4ca95412db65788d16efc04c8', 'request_id': 'req-bec0c362-09c5-421e-8ff7-d0947adc9a87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.517067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:57.533206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] The token cache is being invalidate because project 87e693b4ca95412db65788d16efc04c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:57.533802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.534661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bec0c362-09c5-421e-8ff7-d0947adc9a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.536992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 990/1981] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:57 2022] DELETE /identity/v3/projects/87e693b4ca95412db65788d16efc04c8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:57.542339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42405840-08d2-4d95-808e-843b0b89edf1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.543524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42405840-08d2-4d95-808e-843b0b89edf1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.544331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42405840-08d2-4d95-808e-843b0b89edf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.568809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-42405840-08d2-4d95-808e-843b0b89edf1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:57.570074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-42405840-08d2-4d95-808e-843b0b89edf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.611085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def146d6-91b8-4bae-8143-7e18cada0703 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.685010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 991/1982] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:57.685267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.686099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.686384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.686793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] PATH_INFO: `/v3/users/75391c2757574bea8e9704b3398978c6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.689551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75391c2757574bea8e9704b3398978c6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.691389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.691618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.692083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.692513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.727427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:57.728486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.740302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 992/1983] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:57.744733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.760146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ac73-e687-4be9-945c-abb844800bd8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.769158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.770051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.770498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.770906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.771699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:57.773317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.775825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.790518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 993/1984] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:57.794983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75391c2757574bea8e9704b3398978c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.795632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75391c2757574bea8e9704b3398978c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.802073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.816297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75391c2757574bea8e9704b3398978c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.816917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2acc6c-5852-4002-a47a-fd5e098bf4e8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:57.817378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75391c2757574bea8e9704b3398978c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.826527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:57.827184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:57.827468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:57.827747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:57.828402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:57.830339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:57.834075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75391c2757574bea8e9704b3398978c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:57.850537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 992/1985] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:24:57 2022] DELETE /identity/v3/users/75391c2757574bea8e9704b3398978c6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:57.868290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 994/1986] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:57 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:57.883635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:57.885414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:57.944580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:57.999406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21080-c1bd-4278-bd2e-9db83798b30d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.123610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.124232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.124633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.124908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.126183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.130182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.180729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 993/1987] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:57 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:58.185173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.203111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555c41-5567-4314-beae-1996598a5e0b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.209484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.210431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.211288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.211881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.212727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.216892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.229929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 995/1988] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:58.235497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.255278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4574682-f422-4215-b2b7-4e145928b013 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.264182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.266786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.267067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.267339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.268841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.270608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.272329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.275056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.312027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 994/1989] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:58.312774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.320172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.339403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b85e-2747-45e9-b831-8f81f3e12c42 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.346841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.347571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.348174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.348540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] PATH_INFO: `/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.350113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b90d343c60848e1bab2f9d16b5e8a35)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.352188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.382563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e9196-a833-42c9-922e-8d83d88d8dfc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.388258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.389168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.390281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.390713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.400601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.442790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:58.462436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b90d343c60848e1bab2f9d16b5e8a35', 'request_id': 'req-dd46d687-670e-4f56-b7b0-3aa3462f7ced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:58.477795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b90d343c60848e1bab2f9d16b5e8a35', 'request_id': 'req-dd46d687-670e-4f56-b7b0-3aa3462f7ced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:58.499027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b90d343c60848e1bab2f9d16b5e8a35', 'request_id': 'req-dd46d687-670e-4f56-b7b0-3aa3462f7ced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:58.520860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] The token cache is being invalidate because project 8b90d343c60848e1bab2f9d16b5e8a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:24:58.521359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:58.521933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:24:58.523481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 995/1990] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:24:58 2022] DELETE /identity/v3/projects/8b90d343c60848e1bab2f9d16b5e8a35 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:58.528226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.528226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.541626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.543649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc4ba5-5a4a-42a9-90b5-327edd22b378 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.572257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 996/1991] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:24:58.573658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9bd14-3343-40f9-87c9-1ab7f0301598 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.574005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9bd14-3343-40f9-87c9-1ab7f0301598 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.575914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9bd14-3343-40f9-87c9-1ab7f0301598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.607151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-74e9bd14-3343-40f9-87c9-1ab7f0301598 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:58.609975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-74e9bd14-3343-40f9-87c9-1ab7f0301598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.610096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:58.675299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d687-670e-4f56-b7b0-3aa3462f7ced admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.680689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.681517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.681841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.682290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] PATH_INFO: `/v3/projects/13299687dd6f472395f437c781aa1670/users/15fed1c81347450bb9577d8c77f66019/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.717900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 997/1992] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:58.723543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13299687dd6f472395f437c781aa1670, user_id=15fed1c81347450bb9577d8c77f66019, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.736971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2ad6b-13d0-4828-8991-7a08a1919230 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.742163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.742567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.742968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.747177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 996/1993] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:58 2022] PUT /identity/v3/projects/13299687dd6f472395f437c781aa1670/users/15fed1c81347450bb9577d8c77f66019/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:58.758494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.758779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.759051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.766331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:58.767538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.788609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] MFA Rules not processed for user `15fed1c81347450bb9577d8c77f66019`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:58.790385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:58.852515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 998/1994] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:58.865755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.877261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52977adf-6d3d-4d56-9436-098e914f720f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.885389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 997/1995] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:58.886530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.887387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.888088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.888441 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.888789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.889553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:24:58.889951 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.891660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.903015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d881d9c-f1c9-4f84-af8e-c68bd5716426 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.906831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 999/1996] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:24:58.910090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.910903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.911013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.911248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.912185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.912738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:58.914843 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.955832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2b67-068c-4517-81f3-947249a1c314 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:58.962505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:58.963747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:58.964131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:58.964409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:58.965827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:58.968455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:58.973448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 998/1997] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:58.976892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.004815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1000/1998] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:59.006880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.039416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:59.074197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d3c56-6a12-43c2-a30e-607b68e57582 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.082780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.083525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.083857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.084159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.084862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.086995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.096508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef43e7-a184-42e5-ae60-d42b0ac6988a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.102660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.103165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.103480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.103716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.104480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.106941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.110818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 999/1999] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:58 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:59.114502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.128425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af68a-5f17-4341-a586-8b5b6d59aec4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.133336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.133934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.134209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.134583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.135230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.137129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.140676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1001/2000] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:59 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:59.141690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.148540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1000/2001] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:59.154274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.155291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574a54f-e74a-49a5-a91a-22f68d013297 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.160324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.161047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.161380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.161668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.162306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.164197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.166584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a8a3e-5be7-4bfd-b051-21f020880796 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.172104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.172924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.173232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.176995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1002/2002] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:59.178256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] PATH_INFO: `/v3/projects/ca26e59282f241d2b23ba846da134536/users/aec8f83fed774e30bd64b188b12b8f7f/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.180253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.189117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:59.192387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e3a19-56d2-465e-90ce-19588e0678d4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.196675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca26e59282f241d2b23ba846da134536, user_id=aec8f83fed774e30bd64b188b12b8f7f, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.197687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.198309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.198591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.198973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.200064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.200280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.201730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.212865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1001/2003] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:59 2022] PUT /identity/v3/projects/ca26e59282f241d2b23ba846da134536/users/aec8f83fed774e30bd64b188b12b8f7f/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:59.216974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1003/2004] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:59.217560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.219052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.228656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588501d-066d-4f7b-bc5e-ccbf608c0a3a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.280406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910e3b7-8121-4705-8282-50ace9c81308 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.285338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.286103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.286216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.286646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.286861 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.286996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.287139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.287412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] PATH_INFO: `/v3/projects/9c4d15a1ce32406e94feb1f40cdcd8f8/users/a10ea5bc342e4f76851f9fdcf7759b61/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.287886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.289628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.303610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1002/2005] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:24:59.303781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4d15a1ce32406e94feb1f40cdcd8f8, user_id=a10ea5bc342e4f76851f9fdcf7759b61, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.304611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.306156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510835b7-20b2-4792-b443-c4e430e41cef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.314874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab583c28-da00-4738-863a-c0fa484f0743 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.319784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1004/2006] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:59 2022] PUT /identity/v3/projects/9c4d15a1ce32406e94feb1f40cdcd8f8/users/a10ea5bc342e4f76851f9fdcf7759b61/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:24:59.332601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.332884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.333232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.365043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] MFA Rules not processed for user `a10ea5bc342e4f76851f9fdcf7759b61`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:59.366462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:59.375326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.375988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.376267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.376543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] PATH_INFO: `/v3/projects/ca26e59282f241d2b23ba846da134536/users/aec8f83fed774e30bd64b188b12b8f7f/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.378501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca26e59282f241d2b23ba846da134536, user_id=aec8f83fed774e30bd64b188b12b8f7f, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.381171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835436e3-464a-4dfa-b963-9545991696a3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.402654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1003/2007] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:59 2022] PUT /identity/v3/projects/ca26e59282f241d2b23ba846da134536/users/aec8f83fed774e30bd64b188b12b8f7f/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:24:59.404489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.404779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.405086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.438570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] MFA Rules not processed for user `aec8f83fed774e30bd64b188b12b8f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:24:59.439848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:59.457015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1005/2008] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:24:59.460116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.474693 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:59.486003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfea79-b8f7-47a0-9bbb-7e9806fe4903 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.537541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1004/2009] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:24:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:24:59.550726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.552255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:24:59.566144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:24:59.592449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.593448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.593885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.594258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.595047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.597712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.627968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0156b8-e857-40bc-9801-d400c859457a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.634625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.636020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.636315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.637222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.643645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1006/2010] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:24:59 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:59.645574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.666150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.672198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.699878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1005/2011] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:24:59.812008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31e055-8139-450f-8648-be37ec93ca2d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.817080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.834814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.835595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.835905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.838750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.839600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.844604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867a25-1d8b-4bbe-837f-024ca70a2cd7 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.854473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.915796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1007/2012] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:24:59 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:24:59.920789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:24:59.923725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.924584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.924984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.925427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.933408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea65-18bc-4cd0-8076-8eaad952b43f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:24:59.940041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:24:59.940895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:24:59.941937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:24:59.941937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:24:59.942558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:24:59.944634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:24:59.993452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1008/2013] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:24:59.994628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.010260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40144641-463e-4cc2-b5da-3daa0f8d3cfa admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.024828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.025532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.025812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.026088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] PATH_INFO: `/v3/projects/453b988e369c44b484de8e860afc4d7d/users/82b664bd5ddc4c31b6ca93e099777180/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.044501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.046346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.059945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=453b988e369c44b484de8e860afc4d7d, user_id=82b664bd5ddc4c31b6ca93e099777180, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.063124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.080896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1006/2014] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:24:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:00.092589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1009/2015] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:24:59 2022] PUT /identity/v3/projects/453b988e369c44b484de8e860afc4d7d/users/82b664bd5ddc4c31b6ca93e099777180/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:00.096616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.121259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaa05-683c-4469-a25b-17489307b3c5 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.264150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.266258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.266712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.267057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.268355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.271423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217801a6-2ff0-4f21-bb2c-7b1116c3bed9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.273066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:00.288288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1007/2016] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:00.292213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.311256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7cba5-c768-4730-bd9b-7dea91ef76cb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.325602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.327588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.329222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.329884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] PATH_INFO: `/v3/projects/453b988e369c44b484de8e860afc4d7d/users/82b664bd5ddc4c31b6ca93e099777180/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.334207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] RBAC: Authorizing `identity:create_grant(project_id=453b988e369c44b484de8e860afc4d7d, user_id=82b664bd5ddc4c31b6ca93e099777180, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.343465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.375722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1010/2017] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:00 2022] PUT /identity/v3/projects/453b988e369c44b484de8e860afc4d7d/users/82b664bd5ddc4c31b6ca93e099777180/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:00.387802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.388611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.388970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.465918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] MFA Rules not processed for user `82b664bd5ddc4c31b6ca93e099777180`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:00.468766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:00.599983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1008/2018] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:00.612800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.649477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.650952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:00.656024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:00.684133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dc93-a8ef-43ab-bb2d-6c424b47577a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.688462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.689112 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.689388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.689666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.699574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.701044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.714996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1011/2019] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:00.731601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c99dfb-5eef-4418-9680-b01e886a5c2c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.736178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.736806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.737108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.737380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] PATH_INFO: `/v3/users/a47410e2865a4243b4a1b2cef8325643` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.738517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47410e2865a4243b4a1b2cef8325643)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.740416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.791356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47410e2865a4243b4a1b2cef8325643'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:00.792323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47410e2865a4243b4a1b2cef8325643'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:00.801989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47410e2865a4243b4a1b2cef8325643'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:00.802796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47410e2865a4243b4a1b2cef8325643'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:00.816867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.819987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cd5ccfc2-2153-4686-b9e4-aba02875021f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47410e2865a4243b4a1b2cef8325643'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:00.826339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e89d4-8b28-4afa-b8a5-6daa038fbf65 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:00.830257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:00.830889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:00.831063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.831325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.831703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.833627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:00.838758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1009/2020] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:00 2022] DELETE /identity/v3/users/a47410e2865a4243b4a1b2cef8325643 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:00.845804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:00.846498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:00.847137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:00.874546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:00.875763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:00.909168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:00.910767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:00.920810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1012/2021] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:00.949074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1010/2022] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:00.960403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:00.961906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:01.047178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397808ab-6be1-415d-93fe-e01810b598d5 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:01.110535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:01.111160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:01.111435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:01.111700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:01.124071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:01.126603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8e92-13db-4aee-9025-89b07d600a3d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:01.140996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1013/2023] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:01.325636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:01.327054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:01.343357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0e7a6-e32d-4391-9eec-330b9f59e67a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:01.350702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:01.351953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:01.352131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:01.352434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] PATH_INFO: `/v3/users/a10ea5bc342e4f76851f9fdcf7759b61` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:01.353944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10ea5bc342e4f76851f9fdcf7759b61)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:01.357774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:01.492374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10ea5bc342e4f76851f9fdcf7759b61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:01.493177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10ea5bc342e4f76851f9fdcf7759b61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:01.513473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10ea5bc342e4f76851f9fdcf7759b61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:01.514526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10ea5bc342e4f76851f9fdcf7759b61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:01.546992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10ea5bc342e4f76851f9fdcf7759b61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:01.597339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1011/2024] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:01 2022] DELETE /identity/v3/users/a10ea5bc342e4f76851f9fdcf7759b61 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:01.616519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:01.617033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:01.618456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:01.659491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:01.661245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:01.706528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:01.797885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1014/2025] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:01.817896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:01.819861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:01.903146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:01.905334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:01.959236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735c4e6-3ab3-4c1e-93db-d1d0226a4806 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:01.969144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfafb1-2e1e-4ef8-af9c-6e0f873685b4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:01.974251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:01.974982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:01.975356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:01.975721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:01.978077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:01.978734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:01.979008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:01.979280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] PATH_INFO: `/v3/projects/16ec42b66c0c4c85bbaba6d624b309aa` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:01.983319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ec42b66c0c4c85bbaba6d624b309aa)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:01.986481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:01.990252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:01.992457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:02.015897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1012/2026] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:02.066641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ec42b66c0c4c85bbaba6d624b309aa', 'request_id': 'req-618367b5-bcc5-484a-a175-17b5f259a67c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.076185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ec42b66c0c4c85bbaba6d624b309aa', 'request_id': 'req-618367b5-bcc5-484a-a175-17b5f259a67c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.090645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ec42b66c0c4c85bbaba6d624b309aa', 'request_id': 'req-618367b5-bcc5-484a-a175-17b5f259a67c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.122532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] The token cache is being invalidate because project 16ec42b66c0c4c85bbaba6d624b309aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:02.123152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.125283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.128458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1015/2027] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:01 2022] DELETE /identity/v3/projects/16ec42b66c0c4c85bbaba6d624b309aa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:02.133138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:02.158957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf010bf-0f1a-40a9-b8bd-7051d7e873f7 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:02.177824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:02.249056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:02.249840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:02.250349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:02.250818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] PATH_INFO: `/v3/users/e0a1ed1b2a33498d9222f2de20bbedc2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:02.252157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a1ed1b2a33498d9222f2de20bbedc2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:02.254632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:02.306056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a1ed1b2a33498d9222f2de20bbedc2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.306923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a1ed1b2a33498d9222f2de20bbedc2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.318802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a1ed1b2a33498d9222f2de20bbedc2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.319724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a1ed1b2a33498d9222f2de20bbedc2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.335641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a1ed1b2a33498d9222f2de20bbedc2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.350196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1013/2028] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:02 2022] DELETE /identity/v3/users/e0a1ed1b2a33498d9222f2de20bbedc2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:02.689572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:02.692531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:02.726584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618367b5-bcc5-484a-a175-17b5f259a67c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:02.811090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:02.812103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:02.812315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:02.812650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] PATH_INFO: `/v3/projects/9c4d15a1ce32406e94feb1f40cdcd8f8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:02.813930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4d15a1ce32406e94feb1f40cdcd8f8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:02.818747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:02.819853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:02.821656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:02.838466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6ce7-9170-4b80-9e51-aa85ccad97ae admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:02.863465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:02.864796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:02.869050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:02.869352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] PATH_INFO: `/v3/projects/6a085ca6fd6645b281410787093d06dd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:02.871782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a085ca6fd6645b281410787093d06dd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:02.873865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:02.899501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4d15a1ce32406e94feb1f40cdcd8f8', 'request_id': 'req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.908241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:02.909539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4d15a1ce32406e94feb1f40cdcd8f8', 'request_id': 'req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.921038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4d15a1ce32406e94feb1f40cdcd8f8', 'request_id': 'req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.936771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] The token cache is being invalidate because project 9c4d15a1ce32406e94feb1f40cdcd8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:02.936894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.938021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.939687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1016/2029] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:02 2022] DELETE /identity/v3/projects/9c4d15a1ce32406e94feb1f40cdcd8f8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:02.943494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:02.944865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:02.951852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a085ca6fd6645b281410787093d06dd', 'request_id': 'req-b2751bc7-2abf-4e35-b155-7dac5204df20'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.962515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a085ca6fd6645b281410787093d06dd', 'request_id': 'req-b2751bc7-2abf-4e35-b155-7dac5204df20'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:02.975450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a085ca6fd6645b281410787093d06dd', 'request_id': 'req-b2751bc7-2abf-4e35-b155-7dac5204df20'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.005013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] The token cache is being invalidate because project 6a085ca6fd6645b281410787093d06dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:03.005604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.006621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2751bc7-2abf-4e35-b155-7dac5204df20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.008922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1014/2030] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:02 2022] DELETE /identity/v3/projects/6a085ca6fd6645b281410787093d06dd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:03.014536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfd83-9cf6-457f-a8e3-4162b6bdc356 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.015031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfd83-9cf6-457f-a8e3-4162b6bdc356 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.015471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfd83-9cf6-457f-a8e3-4162b6bdc356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.040053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d2dfd83-9cf6-457f-a8e3-4162b6bdc356 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:03.041286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d2dfd83-9cf6-457f-a8e3-4162b6bdc356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.044555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0ce73-edef-4ce4-b47b-f6b40a20b9e2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.056478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:03.133773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.134862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.135465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.136055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] PATH_INFO: `/v3/users/82b664bd5ddc4c31b6ca93e099777180` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.138249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b664bd5ddc4c31b6ca93e099777180)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.141083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.179228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1015/2031] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:03.188151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.188648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.189185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.215419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:03.216768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.223227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b664bd5ddc4c31b6ca93e099777180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.223882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b664bd5ddc4c31b6ca93e099777180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.224670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b664bd5ddc4c31b6ca93e099777180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.225534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1016/2032] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:03.229505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.231474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.233711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b664bd5ddc4c31b6ca93e099777180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.243067 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511ad42-17cf-4d5e-9880-9e19793411fe None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.247559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b664bd5ddc4c31b6ca93e099777180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.250292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.250997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.251274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.251546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] PATH_INFO: `/v3/users/9e2de5c5b5334744a739f7e1ab7d11c3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.252899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2de5c5b5334744a739f7e1ab7d11c3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.265259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1017/2033] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:02 2022] DELETE /identity/v3/users/82b664bd5ddc4c31b6ca93e099777180 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:03.265518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.266209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.267760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.284618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:03.356149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2de5c5b5334744a739f7e1ab7d11c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.357055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2de5c5b5334744a739f7e1ab7d11c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.366937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2de5c5b5334744a739f7e1ab7d11c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.367790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2de5c5b5334744a739f7e1ab7d11c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.396886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d84-3f6d-4425-857b-1f5558a7d6c1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.399633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54ae1125-ea5c-4a3d-a323-c038705507ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2de5c5b5334744a739f7e1ab7d11c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:03.411225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.413552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.414969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.415418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.417456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:03.419637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.422315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.429250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1017/2034] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:03 2022] DELETE /identity/v3/users/9e2de5c5b5334744a739f7e1ab7d11c3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:03.437374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.437775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.438700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.450486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1018/2035] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:03.456796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.459004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.479678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bbb77-6344-4e7d-981d-40bb590cd230 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.493524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:03.494742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.557768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.559080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.559578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.560170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.561512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.565124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.597943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1018/2036] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:03.621290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1019/2037] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:03 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:03.622764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.641058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.643604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.644246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b94c3-750a-42c4-a384-38b744a22e59 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.731103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:03.807970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c0748-68a7-4918-b517-629cecaebcc1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.813992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.814729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.815036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.815350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.850606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.861797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.862087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.862418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.863093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.865245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.922091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1019/2038] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:03 2022] POST /identity/v3/users => generated 295 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:03.922326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:03.926168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:03.927808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:03.953901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcc82b-7c64-4c8d-b093-929a0e54b817 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:03.965707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.967504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.971323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:03.971992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:03.972278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:03.972555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] PATH_INFO: `/v3/projects/453b988e369c44b484de8e860afc4d7d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:03.973824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=453b988e369c44b484de8e860afc4d7d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:03.975770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:03.990915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1020/2039] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:03.991512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.019873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735f3c-fcee-45f0-ad74-ee1df6477641 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.036460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.038782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.038782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.038782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.039399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.041705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.062873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1021/2040] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:04.068294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.091524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453b988e369c44b484de8e860afc4d7d', 'request_id': 'req-18ca37db-77a7-428e-b087-72921432f74f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.096319 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f68b8-ed6e-46b0-96c9-57b883e38f78 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.099573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453b988e369c44b484de8e860afc4d7d', 'request_id': 'req-18ca37db-77a7-428e-b087-72921432f74f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.111681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453b988e369c44b484de8e860afc4d7d', 'request_id': 'req-18ca37db-77a7-428e-b087-72921432f74f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.139226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] The token cache is being invalidate because project 453b988e369c44b484de8e860afc4d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:04.140711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.143043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-18ca37db-77a7-428e-b087-72921432f74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.148573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1020/2041] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:03 2022] DELETE /identity/v3/projects/453b988e369c44b484de8e860afc4d7d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:04.156184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc46f6-2406-4598-8a15-de1a2f742bd4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.156975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc46f6-2406-4598-8a15-de1a2f742bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.157273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc46f6-2406-4598-8a15-de1a2f742bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.194625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.195327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.195607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.195876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.196582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.198429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.202796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-14bc46f6-2406-4598-8a15-de1a2f742bd4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:04.204052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-14bc46f6-2406-4598-8a15-de1a2f742bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.218646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1022/2042] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:04.219793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.222124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.237265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1021/2043] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:04.239283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.239584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.242093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.242865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e973c-3d18-4a2a-9c1c-043d3748096c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.248661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.249566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.250356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.250712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] PATH_INFO: `/v3/projects/925e39f1b9dc4f86ae09613f997379ed/users/da7672010247460380aaf82c4b8a9966/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.272960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925e39f1b9dc4f86ae09613f997379ed, user_id=da7672010247460380aaf82c4b8a9966, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.277791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226089a5-5448-4ebf-b17c-9e9990ef5831 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.284914 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:04.288409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.304644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1022/2044] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:04.310034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.312409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1023/2045] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:04 2022] PUT /identity/v3/projects/925e39f1b9dc4f86ae09613f997379ed/users/da7672010247460380aaf82c4b8a9966/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:04.322904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.323323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.323886 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.331519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef57cf-3615-4c2e-b434-69ce0281c245 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.394495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] MFA Rules not processed for user `da7672010247460380aaf82c4b8a9966`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:04.395555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.467215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.468510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.468933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.469445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.470130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:04.470466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.474387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.500424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1023/2046] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:04.504506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.538032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8232eb-c487-4ea5-bcb6-642dc8e4ed4b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.554905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.555812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.557061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.557414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.558602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.560546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.581235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1024/2047] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:04.586943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.589270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:04.605918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544cb4f-1eac-4374-9fd6-97dc0b732ba2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.615960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1024/2048] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:04.620096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.623840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.624478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.624761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.625686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.626716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.632026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.706725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1025/2049] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:04.707721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.718528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2ecf-4e26-4955-b23e-254358f9e2a1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.722677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:04.789263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5e5d-b6dc-420c-b751-72d5c8cf6f68 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.795021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.795709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.795984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.796253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.796895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.798871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.809859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.810467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.810746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.811005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.811635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.813960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.826310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1026/2050] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/users => generated 301 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:04.826310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.826310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.838135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfefd2-410a-446b-9528-17cc01c4fff2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.843615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1025/2051] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:04 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:04.843615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.843826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.843826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.843826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.844094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] PATH_INFO: `/v3/projects/3104f093476346d4889c945c362a6a8e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.845220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3104f093476346d4889c945c362a6a8e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.847133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.851269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0575476-e19b-4e74-8ac3-7452b9d4784e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.858438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.859037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.859488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.859769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.860415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.869414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.883217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1026/2052] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:04.884060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.894254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3939ba-a3ee-49bb-a997-c2c0cbd7da4d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.902841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3104f093476346d4889c945c362a6a8e', 'request_id': 'req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.908628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3104f093476346d4889c945c362a6a8e', 'request_id': 'req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.916976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3104f093476346d4889c945c362a6a8e', 'request_id': 'req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.929576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] The token cache is being invalidate because project 3104f093476346d4889c945c362a6a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:04.929867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.930830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:04.932461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1027/2053] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:04 2022] DELETE /identity/v3/projects/3104f093476346d4889c945c362a6a8e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:04.933853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.935497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.954844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.955439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.955710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.955975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.956610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.958223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:04.968446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1027/2054] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:04.969302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:04.971383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:04.983873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924389b0-9168-4a76-97a6-14aeee4918ab admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:04.988714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:04.989431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:04.989785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:04.990123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] PATH_INFO: `/v3/users/994427ebbf5e44838a63dcc9facec834` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:04.991331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994427ebbf5e44838a63dcc9facec834)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:04.993341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.005217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8ca88-5e44-4bc6-b88b-f75c7c2ed47b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.006551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.010558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.011785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.012140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.012556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] PATH_INFO: `/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4/users/0f7df99e43da4aa2b4c470df6dd24869/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.031590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2d2b400b5249d2bcfda9f8c0e809c4, user_id=0f7df99e43da4aa2b4c470df6dd24869, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.034279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.045628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994427ebbf5e44838a63dcc9facec834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.046398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994427ebbf5e44838a63dcc9facec834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.050472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1028/2055] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:04 2022] PUT /identity/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4/users/0f7df99e43da4aa2b4c470df6dd24869/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:05.051308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.054330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.058963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994427ebbf5e44838a63dcc9facec834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.059595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994427ebbf5e44838a63dcc9facec834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.061315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.076510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994427ebbf5e44838a63dcc9facec834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.089679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1028/2056] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:04 2022] DELETE /identity/v3/users/994427ebbf5e44838a63dcc9facec834 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:05.090639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.092002 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.133565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3786-0820-4e8e-baa4-ef9a367fa02f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.171848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809a1d4-e002-4409-a03f-d6a110a4ac64 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.178139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.178579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.178866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.179139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.179797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.181715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.192318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1029/2057] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:05.196869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.203801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.204414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.204696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.204982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.205634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.207225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.208321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b353c3-5c5a-4f2b-93f6-b2a8eb7dccb3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.214155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.214155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.215028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.215297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] PATH_INFO: `/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4/users/0f7df99e43da4aa2b4c470df6dd24869/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.217359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2d2b400b5249d2bcfda9f8c0e809c4, user_id=0f7df99e43da4aa2b4c470df6dd24869, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.219998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b9b65-ea18-42df-8438-aa421d0adb36 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.222948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1029/2058] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:05.224284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.225713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.233156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1030/2059] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:05 2022] PUT /identity/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4/users/0f7df99e43da4aa2b4c470df6dd24869/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:05.241523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d73062-f176-4baf-b377-bb5e990252bc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.244501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.244798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.245115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.278299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] MFA Rules not processed for user `0f7df99e43da4aa2b4c470df6dd24869`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:05.279886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.312792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.313497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.313770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.314321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] PATH_INFO: `/v3/projects/d893423c35cf4c4d91d98408e612f30d/users/60e8e388726b479dbdd013da9efc3cd7/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.334437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d893423c35cf4c4d91d98408e612f30d, user_id=60e8e388726b479dbdd013da9efc3cd7, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.337293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740d30e-73c9-4a0e-b511-bc9680f8690d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.354323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1030/2060] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:05 2022] PUT /identity/v3/projects/d893423c35cf4c4d91d98408e612f30d/users/60e8e388726b479dbdd013da9efc3cd7/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:05.366507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.366771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.367154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.380465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.397671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1031/2061] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:05.398623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.402693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.409674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] MFA Rules not processed for user `60e8e388726b479dbdd013da9efc3cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:05.411347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.423955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695370-85fb-45b3-8d79-595b4c9b3e90 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.448038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.515513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1031/2062] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:05.516990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.518143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.518255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.518356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.518356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] PATH_INFO: `/v3/projects/c30e6035ff924cee80f8b865fe6d9f99` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.522333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.526089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30e6035ff924cee80f8b865fe6d9f99)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.528306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.595895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00dcc-0d35-41f2-8c16-e20d42872a2f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.607918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30e6035ff924cee80f8b865fe6d9f99', 'request_id': 'req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.621379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30e6035ff924cee80f8b865fe6d9f99', 'request_id': 'req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.625180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30e6035ff924cee80f8b865fe6d9f99', 'request_id': 'req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.638357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] The token cache is being invalidate because project c30e6035ff924cee80f8b865fe6d9f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:05.638643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.639647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.642477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1032/2063] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:05 2022] DELETE /identity/v3/projects/c30e6035ff924cee80f8b865fe6d9f99 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:05.642477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.643622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.649700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.650313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.650594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.651032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.652858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.717508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c9ee8-58b3-4499-83d2-be0d04f7d0ce admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.723817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.724541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.724881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.725276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.727222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.731705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.733450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.747065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1032/2064] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:05.747932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.749745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.787856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.795837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:05.820093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.822283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.849151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1033/2065] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:05.859377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d60ae-e56f-4816-a929-f5fde738d16b service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.864094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.864718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.865031 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.865305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] PATH_INFO: `/v3/users/b2102718fbef4f499768ea6f81f2c75d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.866460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2102718fbef4f499768ea6f81f2c75d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.868370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.873969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.875441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.891261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d5bfe-48a3-4087-ad9b-b88e7ad9e156 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:05.898405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:05.899148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:05.899530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:05.899974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] PATH_INFO: `/v3/users/60e8e388726b479dbdd013da9efc3cd7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:05.901458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e8e388726b479dbdd013da9efc3cd7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:05.908329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:05.933695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2102718fbef4f499768ea6f81f2c75d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.934400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2102718fbef4f499768ea6f81f2c75d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.935049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2102718fbef4f499768ea6f81f2c75d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.944370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2102718fbef4f499768ea6f81f2c75d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.959392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2102718fbef4f499768ea6f81f2c75d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.973211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1033/2066] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:05 2022] DELETE /identity/v3/users/b2102718fbef4f499768ea6f81f2c75d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:05.974098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:05.975469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:05.980024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e8e388726b479dbdd013da9efc3cd7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.981192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e8e388726b479dbdd013da9efc3cd7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.989530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e8e388726b479dbdd013da9efc3cd7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:05.990628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e8e388726b479dbdd013da9efc3cd7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.006656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c4ec093-ecf2-4b99-b3d2-64a07ca9a58d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e8e388726b479dbdd013da9efc3cd7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.024442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1034/2067] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:05 2022] DELETE /identity/v3/users/60e8e388726b479dbdd013da9efc3cd7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:06.029197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.029553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.029906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.050918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73992c9b-dada-43c3-b941-d73686d66c4d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.052360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.053889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.106845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:06.130749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.131864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.132295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.132738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] PATH_INFO: `/v3/users/da7672010247460380aaf82c4b8a9966` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.134298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7672010247460380aaf82c4b8a9966)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.137195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.146111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1035/2068] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:06.155909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.158415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.194279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7672010247460380aaf82c4b8a9966'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.195414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7672010247460380aaf82c4b8a9966'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.196862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7672010247460380aaf82c4b8a9966'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.205754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7672010247460380aaf82c4b8a9966'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.220470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-deb26428-eefd-45c7-bcfb-ca706d2f4557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7672010247460380aaf82c4b8a9966'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.226288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373d84-94da-4eed-8d10-e3d6e8ef8941 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.230724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.231347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.231813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.232106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.234008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.236199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1034/2069] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:05 2022] DELETE /identity/v3/users/da7672010247460380aaf82c4b8a9966 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:06.240279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.240551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.240967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.250852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.252683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.257512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:06.265873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1036/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:06.266766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.268109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.275864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.277911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.292921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b666a0-30b5-4495-a57c-bb0fdd364fb3 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.350040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1035/2071] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:06.363379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.365532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.379336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.380042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.380446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.381316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] PATH_INFO: `/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.382141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec09e7698f54d97a7b57259d4e4f39e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.384217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.438624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7e1ab-db45-4ec6-bc4f-348749a9dd5f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.453146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec09e7698f54d97a7b57259d4e4f39e', 'request_id': 'req-f351829c-43db-4b2b-88f0-47e848d04f7d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.460733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec09e7698f54d97a7b57259d4e4f39e', 'request_id': 'req-f351829c-43db-4b2b-88f0-47e848d04f7d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.470348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec09e7698f54d97a7b57259d4e4f39e', 'request_id': 'req-f351829c-43db-4b2b-88f0-47e848d04f7d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.478271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:06.485372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] The token cache is being invalidate because project 4ec09e7698f54d97a7b57259d4e4f39e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:06.486229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.487367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f351829c-43db-4b2b-88f0-47e848d04f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.488967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1037/2072] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:06 2022] DELETE /identity/v3/projects/4ec09e7698f54d97a7b57259d4e4f39e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:06.494544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.494947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.495593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.510011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.510627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.510907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.511213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.513714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.518462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.519734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.598078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.600484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398d0c6-6820-43c6-a194-f3a9f1fbdf6f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.601101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1038/2073] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:06.602044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.611521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.624569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1036/2074] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:06.627683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35a749-bce0-4076-af68-ac974aabe930 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.628508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.628818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.629057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.632580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.633220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.633497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.633771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] PATH_INFO: `/v3/projects/d893423c35cf4c4d91d98408e612f30d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.635643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d893423c35cf4c4d91d98408e612f30d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.637792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.667600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:06.695594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.695594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.724353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd893423c35cf4c4d91d98408e612f30d', 'request_id': 'req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.733669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd893423c35cf4c4d91d98408e612f30d', 'request_id': 'req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.749055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd893423c35cf4c4d91d98408e612f30d', 'request_id': 'req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.771569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] The token cache is being invalidate because project d893423c35cf4c4d91d98408e612f30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:06.772517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.774416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b4c0ad0d-d14f-49cf-bc18-fc624a448fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:06.782572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1039/2075] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:06 2022] DELETE /identity/v3/projects/d893423c35cf4c4d91d98408e612f30d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:06.784598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6aee2-9a61-46f9-98c6-cb625ee78c58 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.784889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6aee2-9a61-46f9-98c6-cb625ee78c58 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.785199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6aee2-9a61-46f9-98c6-cb625ee78c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.794916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1037/2076] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:06.802353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.814818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8fb9b-51ee-4c2d-802a-fc433729f2aa None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.822066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.822702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.823034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.823314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.824435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:06.824748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.825332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4cb6aee2-9a61-46f9-98c6-cb625ee78c58 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.826718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.827106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4cb6aee2-9a61-46f9-98c6-cb625ee78c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.835659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1040/2077] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:06.842506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1038/2078] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:06.844204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.844487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.844769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.854956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.877250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15067ac4-e82e-456a-ba92-32e8b70ef7bf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.890336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:06.891618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:06.894526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.895068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.895369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.895586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.896222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.898145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:06.908863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1041/2079] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:06.913487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.951386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c46f5-6a3a-45c4-951c-c0029cb550c2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:06.960040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1039/2080] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:06.961228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:06.976542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:06.977074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:06.977995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:06.979788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:06.981552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:06.982438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:06.985170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.028770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1042/2081] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:25:07.029992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.055276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60a9-94bf-4919-adc7-861c979378c9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.065355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:07.128029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803b01-4878-4aba-a5d9-322092fd6119 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.152770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.153369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.153649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.153928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.154575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.157172 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.205737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1040/2082] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:06 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:07.208489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.209198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.209489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.209744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.210107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.210697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.212627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.249928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2fa11-ca7a-47b5-9bb7-6b029a1e43df admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.267224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.268295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.269055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.269849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.273239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.279035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.280141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1043/2083] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:07 2022] POST /identity/v3/projects => generated 375 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:07.284869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.328196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1041/2084] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:07.329639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.333088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:07.385743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5a75-820c-4d6b-b778-b07b8d5dc12a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.431333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71091aa-e7ed-4f8f-bf6e-28e733b0b54d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.436997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.438424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.438955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.439436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.440174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.442551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.464777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:07.473022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1044/2085] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:07 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:07.473022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.475201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.475992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.476377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.476829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] PATH_INFO: `/v3/projects/925e39f1b9dc4f86ae09613f997379ed` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.479603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925e39f1b9dc4f86ae09613f997379ed)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.486013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afec82-39d7-4e8a-94b4-0f042365ed14 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.486539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.491737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.492596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.493157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.493532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.494269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.495965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.513320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1045/2086] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:07.513320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.522240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00238819-124a-4a27-96e9-edfb28ed1a1c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.528373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.529006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.529298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.529795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.530339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.531923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.544068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1046/2087] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:07.544980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.550344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925e39f1b9dc4f86ae09613f997379ed', 'request_id': 'req-35e09269-01d2-476b-bab7-650b416adf38'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.554627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f81cb-681c-4d2a-b904-686b7424d4f8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.559413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925e39f1b9dc4f86ae09613f997379ed', 'request_id': 'req-35e09269-01d2-476b-bab7-650b416adf38'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.569720 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925e39f1b9dc4f86ae09613f997379ed', 'request_id': 'req-35e09269-01d2-476b-bab7-650b416adf38'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.583879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] The token cache is being invalidate because project 925e39f1b9dc4f86ae09613f997379ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:07.584192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.587908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.587908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1042/2088] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:07 2022] DELETE /identity/v3/projects/925e39f1b9dc4f86ae09613f997379ed => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:07.591046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.592625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:07.617191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.617933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.618276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.618612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] PATH_INFO: `/v3/projects/d098e93cf05542aeb904cd3cffae5c3e/users/ebd672dcb2734baeb353c72a664a77e6/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.641153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d098e93cf05542aeb904cd3cffae5c3e, user_id=ebd672dcb2734baeb353c72a664a77e6, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.644156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.652666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:07.665257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1047/2089] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:07 2022] PUT /identity/v3/projects/d098e93cf05542aeb904cd3cffae5c3e/users/ebd672dcb2734baeb353c72a664a77e6/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:07.665964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.668363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:07.671603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e09269-01d2-476b-bab7-650b416adf38 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.755934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.756867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.757026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415fc0-fbe1-4a56-a38a-c3636ad6ecb8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:07.757175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.757443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.758102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.759894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b320-f06e-4f4d-beaf-b2920f1047d8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.763562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:07.764263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.764556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.764891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] PATH_INFO: `/v3/users/0f7df99e43da4aa2b4c470df6dd24869` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.766103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7df99e43da4aa2b4c470df6dd24869)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:07.768276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:07.776637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1043/2090] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:07.779962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:07.780532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:07.781223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:07.816204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] MFA Rules not processed for user `ebd672dcb2734baeb353c72a664a77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:07.817615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:07.840566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7df99e43da4aa2b4c470df6dd24869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.841749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7df99e43da4aa2b4c470df6dd24869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.852915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7df99e43da4aa2b4c470df6dd24869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.854133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7df99e43da4aa2b4c470df6dd24869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.860658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:07.876528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7df99e43da4aa2b4c470df6dd24869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:07.898614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1048/2091] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:07 2022] DELETE /identity/v3/users/0f7df99e43da4aa2b4c470df6dd24869 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:07.898614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.898614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:07.942634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1044/2092] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:07.952322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:07.954215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.053409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ff627-c239-4908-b338-ccaeca8a172a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:08.077236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37529b1-1b9b-4a46-94be-b7125dea0eb1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:08.089069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:08.092616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.092925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.093231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] PATH_INFO: `/v3/projects/55629177df8e4c80b987d33590338492/users/a38812a433e345499a6a40633b1c7147/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.122773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55629177df8e4c80b987d33590338492, user_id=a38812a433e345499a6a40633b1c7147, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:08.123354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:08.125600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617954c9-469b-478e-93c8-5fa96d6d5061 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:08.144417 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:08.146045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.146489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.146831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.160337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1049/2093] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:07 2022] PUT /identity/v3/projects/55629177df8e4c80b987d33590338492/users/a38812a433e345499a6a40633b1c7147/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:08.180199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.180746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.181315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.248494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] MFA Rules not processed for user `a38812a433e345499a6a40633b1c7147`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:08.249826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.252865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:08.255347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:08.280027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1045/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:08.397564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1050/2095] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:08.403002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:08.405122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.425966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:08.435414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.440580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f788a5-5176-407c-92a1-c1a78cf84ad5 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:08.527428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9caa5-3b55-41ad-a718-4b6cef1ba682 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:08.532053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:08.532799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.533176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.533563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.541479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:08.544709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:08.546227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658b9af-6e56-42fb-9d8e-b10b599a844c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:08.559698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1051/2096] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:08.569251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:08.570377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.570377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.570797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] PATH_INFO: `/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.572203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2d2b400b5249d2bcfda9f8c0e809c4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:08.574907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:08.659224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2d2b400b5249d2bcfda9f8c0e809c4', 'request_id': 'req-8169a369-b0df-40a5-81ea-7d7e46bd37d2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:08.666349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2d2b400b5249d2bcfda9f8c0e809c4', 'request_id': 'req-8169a369-b0df-40a5-81ea-7d7e46bd37d2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:08.679291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2d2b400b5249d2bcfda9f8c0e809c4', 'request_id': 'req-8169a369-b0df-40a5-81ea-7d7e46bd37d2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:08.702410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] The token cache is being invalidate because project 0a2d2b400b5249d2bcfda9f8c0e809c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:08.706569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:08.713480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8169a369-b0df-40a5-81ea-7d7e46bd37d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:08.719979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1046/2097] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:08 2022] DELETE /identity/v3/projects/0a2d2b400b5249d2bcfda9f8c0e809c4 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:08.726958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.727550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.728056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.775108 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:08.777667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.873426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1052/2098] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:08.890258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.890543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.890818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.920183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:08.921566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:08.932439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1047/2099] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:08.937234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:08.948482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08677ebd-8563-4fd4-9552-d63adb12ebb8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:08.955957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:08.957629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:08.957903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:08.958245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:08.959085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:08.959389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:08.961318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.008328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1053/2100] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:09.019616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.035393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7086-8e4d-4461-9b7f-a503861e183e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.052313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.052933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.053225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.053493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.054250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.056278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.116743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:09.131299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.142481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1048/2101] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:09 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:09.147027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f09f75-5c30-4eff-984a-8843624c7587 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.279437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.280155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.280438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.280833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.281679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.284294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.313429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1054/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:09 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:09.317552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.328807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cbbfa-9ea3-4172-9ed6-e0a8574d7911 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.334871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.335617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.335896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.336162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.337350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.339409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.381679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1049/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:25:09.386017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.397931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80695c-6b85-4998-9676-8c78acdc5d66 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.403190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.404215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.404503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.404776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.405455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.407073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db06791-4381-461e-beec-2631389a6e91 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.418355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1055/2104] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:09.422839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.433773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4a4b-2fd0-43a9-a64b-17fa17a9ae49 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.438528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.439169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.439439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.439706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] PATH_INFO: `/v3/projects/3090754f4dd543518b950f21cef74587/users/f280e9b875264544b92c1143086a6be9/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.455140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3090754f4dd543518b950f21cef74587, user_id=f280e9b875264544b92c1143086a6be9, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.457879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.472618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1050/2105] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:09 2022] PUT /identity/v3/projects/3090754f4dd543518b950f21cef74587/users/f280e9b875264544b92c1143086a6be9/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:09.484562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.484853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.485156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.516239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] MFA Rules not processed for user `f280e9b875264544b92c1143086a6be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:09.517535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:09.570346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:09.593831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1056/2106] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:09.602821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:09.604580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:09.691822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4046ba-0e0a-45cc-84ea-ef24d05a93b1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:09.695645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:09.696248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:09.696678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:09.698489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:09.712541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:09.716027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46dcc-69e0-47f1-95e4-b4329852b2f8 service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:09.731858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1051/2107] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:10.629247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:10.630600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:10.762833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3175-bd4d-4567-9822-4c8e3f571f87 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:10.770915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:10.771717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:10.772264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:10.772866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] PATH_INFO: `/v3/users/15fed1c81347450bb9577d8c77f66019` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:10.774179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15fed1c81347450bb9577d8c77f66019)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:10.776252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:10.889644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15fed1c81347450bb9577d8c77f66019'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:10.890389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15fed1c81347450bb9577d8c77f66019'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:10.900389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15fed1c81347450bb9577d8c77f66019'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:10.901326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15fed1c81347450bb9577d8c77f66019'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:10.914721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15fed1c81347450bb9577d8c77f66019'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:10.938870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1057/2108] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:10 2022] DELETE /identity/v3/users/15fed1c81347450bb9577d8c77f66019 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:10.943382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:10.943878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:10.944036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:10.972628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:10.973905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.045577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1052/2109] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:11.056107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:11.060266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.159585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298a5b9-4059-46d7-ac39-cb5a0ec7d388 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:11.165112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:11.167225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:11.169360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:11.169892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:11.191449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:11.191449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4509ff-9a8c-4c84-b0a5-0412863aa1ad service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:11.205724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1058/2110] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:11.381906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:11.383572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.396394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578ac0c-446e-46b1-97b1-92ab6c9384b1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:11.401446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:11.402889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:11.403169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:11.403437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] PATH_INFO: `/v3/users/ebd672dcb2734baeb353c72a664a77e6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:11.404590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd672dcb2734baeb353c72a664a77e6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:11.406589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:11.466916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:11.474911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd672dcb2734baeb353c72a664a77e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.475719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd672dcb2734baeb353c72a664a77e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.476672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd672dcb2734baeb353c72a664a77e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.491804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd672dcb2734baeb353c72a664a77e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.533663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd672dcb2734baeb353c72a664a77e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.549184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1053/2111] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:11 2022] DELETE /identity/v3/users/ebd672dcb2734baeb353c72a664a77e6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:11.553889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:11.554203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:11.554500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:11.586890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:11.588234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.598216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:11.666700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1059/2112] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:11.679164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:11.681354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.746138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:11.750964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:11.802149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84651-7bde-4a09-8afe-adcaa8b9350c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:11.830071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:11.831408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:11.831987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:11.832332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] PATH_INFO: `/v3/projects/13299687dd6f472395f437c781aa1670` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:11.833679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13299687dd6f472395f437c781aa1670)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:11.834216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f730-3cf4-46f4-a088-c90bfb440bbe admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:11.835795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:11.838962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:11.839769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:11.840049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:11.840330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:11.853996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:11.857449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:11.871283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1054/2113] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:11.930664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13299687dd6f472395f437c781aa1670', 'request_id': 'req-fa5101eb-02d1-4b42-b1ff-1606d0f30216'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.940351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13299687dd6f472395f437c781aa1670', 'request_id': 'req-fa5101eb-02d1-4b42-b1ff-1606d0f30216'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.956676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13299687dd6f472395f437c781aa1670', 'request_id': 'req-fa5101eb-02d1-4b42-b1ff-1606d0f30216'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.986787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] The token cache is being invalidate because project 13299687dd6f472395f437c781aa1670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:11.987091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.990010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:11.999827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1060/2114] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:11 2022] DELETE /identity/v3/projects/13299687dd6f472395f437c781aa1670 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:12.001609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:12.014243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7eef4-d846-4157-a128-47c37c539c04 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:12.152770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:12.154013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:12.154900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:12.155786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] PATH_INFO: `/v3/users/aec8f83fed774e30bd64b188b12b8f7f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:12.162406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec8f83fed774e30bd64b188b12b8f7f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:12.166175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:12.279035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec8f83fed774e30bd64b188b12b8f7f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.284148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec8f83fed774e30bd64b188b12b8f7f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.285155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec8f83fed774e30bd64b188b12b8f7f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.301414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec8f83fed774e30bd64b188b12b8f7f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.323234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:12.342655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:12.344164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:12.349338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec8f83fed774e30bd64b188b12b8f7f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.362453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5101eb-02d1-4b42-b1ff-1606d0f30216 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:12.369906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:12.371269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:12.371870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:12.372582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] PATH_INFO: `/v3/users/a38812a433e345499a6a40633b1c7147` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:12.374679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38812a433e345499a6a40633b1c7147)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:12.375785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1055/2115] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:12 2022] DELETE /identity/v3/users/aec8f83fed774e30bd64b188b12b8f7f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:12.378381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:12.468853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38812a433e345499a6a40633b1c7147'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.469740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38812a433e345499a6a40633b1c7147'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.493881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38812a433e345499a6a40633b1c7147'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.505384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38812a433e345499a6a40633b1c7147'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.541272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:12.543498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:12.546511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d2c828af-eeeb-45b0-a15c-e19258ccfe83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38812a433e345499a6a40633b1c7147'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.622735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1061/2116] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:12 2022] DELETE /identity/v3/users/a38812a433e345499a6a40633b1c7147 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:12.630058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625c501-c078-491b-a692-4e6551678afa None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:12.637374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625c501-c078-491b-a692-4e6551678afa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:12.641635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625c501-c078-491b-a692-4e6551678afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:12.690322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9625c501-c078-491b-a692-4e6551678afa None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:12.698294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9625c501-c078-491b-a692-4e6551678afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:12.709829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:12.733108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db3c4-3a8c-4c92-af42-bea235b75e15 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:12.816448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1062/2117] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:12.824424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:12.825662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:12.826193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:12.826755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] PATH_INFO: `/v3/projects/d098e93cf05542aeb904cd3cffae5c3e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:12.828278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d098e93cf05542aeb904cd3cffae5c3e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:12.829259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625c501-c078-491b-a692-4e6551678afa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:12.830662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9625c501-c078-491b-a692-4e6551678afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:12.831177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:12.930368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd098e93cf05542aeb904cd3cffae5c3e', 'request_id': 'req-002ed038-5ffe-4f29-9ffe-fabd22d528f0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.938827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd098e93cf05542aeb904cd3cffae5c3e', 'request_id': 'req-002ed038-5ffe-4f29-9ffe-fabd22d528f0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.941624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625c501-c078-491b-a692-4e6551678afa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:12.950584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd098e93cf05542aeb904cd3cffae5c3e', 'request_id': 'req-002ed038-5ffe-4f29-9ffe-fabd22d528f0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.965967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] The token cache is being invalidate because project d098e93cf05542aeb904cd3cffae5c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:12.966249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.966928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:12.968929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1056/2118] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:12 2022] DELETE /identity/v3/projects/d098e93cf05542aeb904cd3cffae5c3e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:12.969798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:12.971752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:13.023139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.024226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.024710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.025322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.028228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:13.041207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:13.165254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed038-5ffe-4f29-9ffe-fabd22d528f0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.181598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.182500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.183005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.184204 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] PATH_INFO: `/v3/projects/ca26e59282f241d2b23ba846da134536` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.185639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca26e59282f241d2b23ba846da134536)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.187672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.240034 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.241561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c5694-86eb-4a9e-9857-ab97d6e4043d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.252556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1063/2119] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:13.263436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca26e59282f241d2b23ba846da134536', 'request_id': 'req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.272051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca26e59282f241d2b23ba846da134536', 'request_id': 'req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.282207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca26e59282f241d2b23ba846da134536', 'request_id': 'req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.304969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] The token cache is being invalidate because project ca26e59282f241d2b23ba846da134536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:13.304969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.304969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f39134de-2ae5-4fab-b5fd-6e612bcbb1fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.312920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1057/2120] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:12 2022] DELETE /identity/v3/projects/ca26e59282f241d2b23ba846da134536 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:13.313542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.314004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.314454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.339045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:13.341471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:13.356722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:13.449739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.450428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.450859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.455033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1064/2121] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:13.486710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:13.487707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:13.500567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1058/2122] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:13.504653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.518928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35556e2f-6909-4205-aa74-c5424b962d53 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.528281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.529041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.529472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.529800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.534986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:13.535297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.537159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.551753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1065/2123] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:13.557091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.566865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d88bd-ef26-4fba-af1a-928aca6861c4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.572653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.573397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.573820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.574432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.575170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.577181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.613207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1059/2124] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:13 2022] POST /identity/v3/projects => generated 357 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:13.617526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.628367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d5798-2d15-495d-9fe1-a35d3c82f954 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.732658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.734596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:13.741518 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.742238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.742631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.743001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.743811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.746005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.809574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1066/2125] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:13 2022] POST /identity/v3/users => generated 279 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:13.812309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.825361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60520d-1c0b-433f-a8c0-8ff202a36f6b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.834789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.835418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.835696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.836152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.836755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.839004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.843677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68185a7-64ee-4abf-b69e-6ec81ad1b168 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.852923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.853874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.854318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.855317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] PATH_INFO: `/v3/projects/55629177df8e4c80b987d33590338492` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.858261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55629177df8e4c80b987d33590338492)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.860989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1067/2126] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:13.864083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.866633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.881349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932edeae-3a43-4c4d-85b0-07c262873911 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.887913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:13.888975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:13.889424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:13.889849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:13.890890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:13.892669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:13.894306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:13.916375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1068/2127] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:25:13.918645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:13.932048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46ce72-30b5-42a4-a554-4308ae9be398 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:13.951428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55629177df8e4c80b987d33590338492', 'request_id': 'req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.960232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55629177df8e4c80b987d33590338492', 'request_id': 'req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.976199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55629177df8e4c80b987d33590338492', 'request_id': 'req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.992570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] The token cache is being invalidate because project 55629177df8e4c80b987d33590338492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:13.993078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:13.993950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b55cd0f2-be45-4d19-b5a2-ea49921a0a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:14.001020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1060/2128] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:13 2022] DELETE /identity/v3/projects/55629177df8e4c80b987d33590338492 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:14.003390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fd0b5-75fd-472f-97eb-857485575708 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.003968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fd0b5-75fd-472f-97eb-857485575708 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.005572 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fd0b5-75fd-472f-97eb-857485575708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.030149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-424fd0b5-75fd-472f-97eb-857485575708 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:14.032276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-424fd0b5-75fd-472f-97eb-857485575708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.050431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1061/2129] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:14.051977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.052244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.052520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.053140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.054064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.056779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.057251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] PATH_INFO: `/v3/projects/230373331c6a46ffae7b87aaf46d9640/users/092f002e099e41fbbebcad3f13fd8965/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.075588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:14.076642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.083117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230373331c6a46ffae7b87aaf46d9640, user_id=092f002e099e41fbbebcad3f13fd8965, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.086101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1062/2130] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:14.086427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c741295-9571-4a36-9f5a-a8ae83a89ca5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.089035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:14.090723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.101474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb55c-d9e6-412d-9bf5-3f5285c1ee4c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.110377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1069/2131] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:13 2022] PUT /identity/v3/projects/230373331c6a46ffae7b87aaf46d9640/users/092f002e099e41fbbebcad3f13fd8965/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:14.121991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.122949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.123519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.176818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] MFA Rules not processed for user `092f002e099e41fbbebcad3f13fd8965`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:14.178646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.196443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.197356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.197737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.198099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.198890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:14.199293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.200983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.214022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1063/2132] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:25:14.218834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.230781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a81a6-99b4-4767-8b68-96ae55e68db0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.236036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.237296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.237296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.237509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.238606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.240554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.285686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1064/2133] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/projects => generated 399 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:14.288095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.295996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1070/2134] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:14.297099 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.298497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.313147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b643e-59c6-40b9-8187-53c3017243f4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.376027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809f4b4-8f7e-409d-9d3d-e8736b6c901d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.384829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.385502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.385879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.386558 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.387304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.389401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.395352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.396091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.396242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.396650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.397334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.399669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.410844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:14.419365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1065/2135] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:14.430799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.443846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5929-3523-4f0f-9096-560f0a3db2aa admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.452904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1071/2136] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/projects => generated 355 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:14.453231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.453231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.453231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.453231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.453590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.454519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.455713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.475797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1066/2137] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:14.486920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.501819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908558-d3a4-4c47-9252-57c96a06d1b2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.540709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba32b-dfda-4c3e-ae64-8bc8ca3470a8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.547068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.547915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.548295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.548860 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.549531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.551681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.573034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.573596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.574108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.574385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.575065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.575665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:14.578664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.585611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1072/2138] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:14.586875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.590782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1067/2139] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:14.594656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.600594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421e7b9-9f3c-4a8f-9ec7-b6f12c90a7fa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.606688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.607301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.607516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5428a-6c71-47fa-989e-53fd9b10670e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.607632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.607833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.608837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.610295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.613541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.613885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.614265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.614545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] PATH_INFO: `/v3/projects/d68aff1db94f4e01a652cb5c764a2b86/users/d631b10caab847f4bf93c9c02c3eb48b/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.621264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1073/2140] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:14.625483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.633408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68aff1db94f4e01a652cb5c764a2b86, user_id=d631b10caab847f4bf93c9c02c3eb48b, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.635722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d01771-b5c1-4382-aef7-aac628d351d0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.636124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4d32d-bdf0-470e-83e9-114901a999ef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.642700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.643371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.643624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.643895 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] PATH_INFO: `/v3/projects/fe3f03cba7f749a4b92548d68ec3416b/users/69326e71045d463fb332172fbc3c3595/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.661095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1068/2141] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:14 2022] PUT /identity/v3/projects/d68aff1db94f4e01a652cb5c764a2b86/users/d631b10caab847f4bf93c9c02c3eb48b/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:14.664076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3f03cba7f749a4b92548d68ec3416b, user_id=69326e71045d463fb332172fbc3c3595, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.667861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.668843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.670141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.670580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.711728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1074/2142] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:14 2022] PUT /identity/v3/projects/fe3f03cba7f749a4b92548d68ec3416b/users/69326e71045d463fb332172fbc3c3595/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:14.719235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.722964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] MFA Rules not processed for user `d631b10caab847f4bf93c9c02c3eb48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:14.724195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.817302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2fb90-a074-4db0-b756-30a077d8b6fd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.828672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.830644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.830921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.831191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.831844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.833480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.844818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1075/2143] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:14.848835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.875630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1069/2144] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:14.881139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d863a1-79eb-48ad-b73e-26fe20c80ec5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:14.887189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:14.888521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:14.888670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.889337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.889827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.890198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] PATH_INFO: `/v3/projects/fe3f03cba7f749a4b92548d68ec3416b/users/69326e71045d463fb332172fbc3c3595/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.894347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3f03cba7f749a4b92548d68ec3416b, user_id=69326e71045d463fb332172fbc3c3595, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:14.898233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed6bff-58b8-4cb8-b914-bc773333dae0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:14.917032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1076/2145] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:14 2022] PUT /identity/v3/projects/fe3f03cba7f749a4b92548d68ec3416b/users/69326e71045d463fb332172fbc3c3595/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:14.929530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:14.929854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:14.930139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:14.969894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] MFA Rules not processed for user `69326e71045d463fb332172fbc3c3595`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:14.973347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:14.981397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:14.988216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f18ecf-280c-488b-aea4-4f38d8f60448 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:15.059762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:15.060570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:15.061229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:15.061615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:15.086336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1077/2146] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:15.099126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:15.101964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:15.123641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757f6e4-7bd8-4d70-8250-5c06b518be6a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:15.132958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:15.132958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:15.132958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:15.132958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:15.147712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:15.149397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:15.162495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:15.162717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1078/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:15.164373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:15.177492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1070/2148] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:15.257899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:15.268032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b757fd-e8e8-4ebf-9ff2-44fc5089ebbc service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:15.275057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:15.275794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:15.276170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:15.276538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:15.299388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:15.362981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:15.364778 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302bf2c-d05a-46a6-9297-2cfabf77da0c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:15.377912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1079/2149] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:23.469201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:23.530692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a0a8-0fca-41fb-b06a-ea57824dd7fa service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:23.535156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:23.535775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:23.536050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:23.536322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] PATH_INFO: `/v3/users/d631b10caab847f4bf93c9c02c3eb48b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:23.537461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d631b10caab847f4bf93c9c02c3eb48b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:23.539362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:23.586008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd631b10caab847f4bf93c9c02c3eb48b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:23.586758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd631b10caab847f4bf93c9c02c3eb48b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:23.587350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd631b10caab847f4bf93c9c02c3eb48b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:23.595062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd631b10caab847f4bf93c9c02c3eb48b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:23.599455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:23.608834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd631b10caab847f4bf93c9c02c3eb48b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:23.621780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1071/2150] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:23 2022] DELETE /identity/v3/users/d631b10caab847f4bf93c9c02c3eb48b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:23.626419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:23.626892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:23.627029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:23.646133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:23.647323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:23.707803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1080/2151] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:23.718330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:23.719712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:23.775377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f7a49-9580-49b1-8d59-2d15a95c1394 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:23.779600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:23.780199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:23.780473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:23.780740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:23.789788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:23.791189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:23.800646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1072/2152] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:24.076081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:24.077575 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:24.090746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f350-cd05-49a9-9ed5-0f6c9e259bea None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:24.095369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:24.096071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:24.096334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:24.096597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] PATH_INFO: `/v3/projects/d68aff1db94f4e01a652cb5c764a2b86` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:24.097934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68aff1db94f4e01a652cb5c764a2b86)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:24.099856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:24.161852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68aff1db94f4e01a652cb5c764a2b86', 'request_id': 'req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:24.169708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68aff1db94f4e01a652cb5c764a2b86', 'request_id': 'req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:24.178529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68aff1db94f4e01a652cb5c764a2b86', 'request_id': 'req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:24.192206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] The token cache is being invalidate because project d68aff1db94f4e01a652cb5c764a2b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:24.192493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:24.193164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-83344b5b-63aa-4ae5-8bf1-fd32d0bf5ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:24.194937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1081/2153] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:24 2022] DELETE /identity/v3/projects/d68aff1db94f4e01a652cb5c764a2b86 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:32.812731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:32.814310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:32.828243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ab4d-db40-4ebe-8544-dbb964bb70d4 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:32.886771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:32.887437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:32.888803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:32.889169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] PATH_INFO: `/v3/users/092f002e099e41fbbebcad3f13fd8965` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:32.890405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=092f002e099e41fbbebcad3f13fd8965)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:32.892357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:32.944071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092f002e099e41fbbebcad3f13fd8965'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:32.944918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092f002e099e41fbbebcad3f13fd8965'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:32.945604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092f002e099e41fbbebcad3f13fd8965'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:32.953388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092f002e099e41fbbebcad3f13fd8965'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:32.965831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092f002e099e41fbbebcad3f13fd8965'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:32.977600 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:32.982365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1073/2154] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:32 2022] DELETE /identity/v3/users/092f002e099e41fbbebcad3f13fd8965 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:32.986578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:32.987118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:32.987118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:33.007262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:33.008203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:33.042461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:33.073087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1082/2155] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:33.081293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:33.082685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:33.152879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd707ba-d3ad-4d56-908c-76c769853d9d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:33.157313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:33.157925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:33.158194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:33.158460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:33.168642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:33.171166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:33.183188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1074/2156] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:33.695262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:33.704710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:33.733409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d59dd8-d401-437d-8aa6-9298fe9adfc4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:33.738612 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:33.739242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:33.739520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:33.739788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] PATH_INFO: `/v3/projects/230373331c6a46ffae7b87aaf46d9640` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:33.740937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230373331c6a46ffae7b87aaf46d9640)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:33.742868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:33.812121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230373331c6a46ffae7b87aaf46d9640', 'request_id': 'req-2c31fdc5-c119-4482-97f9-befe90e59e32'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:33.818942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230373331c6a46ffae7b87aaf46d9640', 'request_id': 'req-2c31fdc5-c119-4482-97f9-befe90e59e32'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:33.832691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230373331c6a46ffae7b87aaf46d9640', 'request_id': 'req-2c31fdc5-c119-4482-97f9-befe90e59e32'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:33.849764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] The token cache is being invalidate because project 230373331c6a46ffae7b87aaf46d9640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:33.849764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:33.850717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:33.852631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1083/2157] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:33 2022] DELETE /identity/v3/projects/230373331c6a46ffae7b87aaf46d9640 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:33.856661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:33.868069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ae4-80fc-46ac-b0fe-3a38b364cc01 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:33.968071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:33.968724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:33.969059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:33.969829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] PATH_INFO: `/v3/users/69326e71045d463fb332172fbc3c3595` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:33.971391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=69326e71045d463fb332172fbc3c3595)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:33.973871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.053639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69326e71045d463fb332172fbc3c3595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.054980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69326e71045d463fb332172fbc3c3595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.055994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69326e71045d463fb332172fbc3c3595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.073432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69326e71045d463fb332172fbc3c3595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.096752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69326e71045d463fb332172fbc3c3595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.111916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1075/2158] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:33 2022] DELETE /identity/v3/users/69326e71045d463fb332172fbc3c3595 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:34.300187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.301704 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.316722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fdc5-c119-4482-97f9-befe90e59e32 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.385395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.386087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.386358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.386623 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] PATH_INFO: `/v3/users/c0983c1e9f7649c2890000959cea452b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.387804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0983c1e9f7649c2890000959cea452b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.389874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.425091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.426809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.442602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec6599-1907-4555-b8e5-8c806ec6ce2e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.443790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0983c1e9f7649c2890000959cea452b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.444569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0983c1e9f7649c2890000959cea452b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.447819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.448459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.448729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.449018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] PATH_INFO: `/v3/projects/fe3f03cba7f749a4b92548d68ec3416b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.450784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3f03cba7f749a4b92548d68ec3416b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.452239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0983c1e9f7649c2890000959cea452b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.452912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0983c1e9f7649c2890000959cea452b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.453259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.460187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:34.467171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dcc8d0e7-8fab-4233-92f8-67b102543d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0983c1e9f7649c2890000959cea452b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.474518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:34.480838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1084/2159] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:34 2022] DELETE /identity/v3/users/c0983c1e9f7649c2890000959cea452b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:34.517218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3f03cba7f749a4b92548d68ec3416b', 'request_id': 'req-0baa540d-a8c1-4b01-ace1-5249504bfd30'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.524023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3f03cba7f749a4b92548d68ec3416b', 'request_id': 'req-0baa540d-a8c1-4b01-ace1-5249504bfd30'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.532632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3f03cba7f749a4b92548d68ec3416b', 'request_id': 'req-0baa540d-a8c1-4b01-ace1-5249504bfd30'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.544981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] The token cache is being invalidate because project fe3f03cba7f749a4b92548d68ec3416b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:34.545279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.546270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0baa540d-a8c1-4b01-ace1-5249504bfd30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.547472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1076/2160] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:34 2022] DELETE /identity/v3/projects/fe3f03cba7f749a4b92548d68ec3416b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:34.552295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.552585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.552862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.574869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:34.576560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.640196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1085/2161] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:34.650211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.650631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.651067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.670338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.672302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.674991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:34.676307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.688711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1077/2162] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:34.698956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55738b6f-ca3d-4e9d-90f5-00afdcaa9f48 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.704792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.707298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.708180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.708635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.709719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] PATH_INFO: `/v3/projects/416f25107a0d4eb0bab9d92e59cf07f4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.710926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=416f25107a0d4eb0bab9d92e59cf07f4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.712918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.716889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909adb-d36f-4865-b1b3-eb30a21fb113 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.723979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.724589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.724867 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.725181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.725805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:34.726106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.727809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.741667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1078/2163] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:34.746460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.757194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808f59-a8f1-4cf4-b52b-a6c6e131587f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.768815 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '416f25107a0d4eb0bab9d92e59cf07f4', 'request_id': 'req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.774604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '416f25107a0d4eb0bab9d92e59cf07f4', 'request_id': 'req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.783950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '416f25107a0d4eb0bab9d92e59cf07f4', 'request_id': 'req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.802920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] The token cache is being invalidate because project 416f25107a0d4eb0bab9d92e59cf07f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:34.803194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.803857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-48e8b6fd-a487-48ef-9169-b0d27d4e7d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:34.805830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1086/2164] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:34 2022] DELETE /identity/v3/projects/416f25107a0d4eb0bab9d92e59cf07f4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:34.812544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9f50-5cea-4a21-96cc-5ccb97522a6d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.812826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9f50-5cea-4a21-96cc-5ccb97522a6d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.813276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9f50-5cea-4a21-96cc-5ccb97522a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.820734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.821366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.822456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.822456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.822647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.824392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.833433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4c3c9f50-5cea-4a21-96cc-5ccb97522a6d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:34.835029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c3c9f50-5cea-4a21-96cc-5ccb97522a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.843503 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1087/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:34.851460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.851804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.852080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.858915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1079/2166] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:34.862931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.864983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.872853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:34.874079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:34.929844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:34.942058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1088/2167] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:34.945890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.953852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7ad6-a55a-4c4a-8569-0250d2d88199 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.957497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf3e3b-2ba1-4683-ad63-b73218d694c6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:34.959352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.960126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.960501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.960874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.961640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.964461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:34.964461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:34.964461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:34.964767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.964871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:34.964871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:34.964871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:34.969880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:34.984699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1089/2168] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:25:34.986946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:34.989735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:34.994987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1080/2169] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:35.001261 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.004398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12d3b-988d-42b9-8a25-ef3cb697a5e6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.010184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.010789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.011578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.011578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.012058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.014449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.021246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5a1f-2e45-4e84-b31c-276d16a1d291 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.036285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.037156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.037648 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.038069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.038870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.041525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.047964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1090/2170] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:34 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:35.052048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.055497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1081/2171] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:35.061285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.072664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb07db9-6931-46c4-a362-16a3c7133d30 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.129745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b828165-6fe5-44b4-91db-976932ecd749 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.134609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.135025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.135218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.135493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.135629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.135761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.135906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.136180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.136420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.136821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.138465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.138570 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.149151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1082/2172] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:35.153868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.163077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1091/2173] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:35.164591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4cf2-33ed-44dd-bfbd-e3b69b4977a7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.167194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.170317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.171102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.171383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.171657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] PATH_INFO: `/v3/projects/87c6aa7009f64531bfaf29fac8f1286b/users/361c9d3f558c4abf9a1c21095a30876f/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.177802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bcadd-9251-4a9a-9007-42ab1276d465 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.182697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.183279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.183552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.183822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.184501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.186489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.187687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c6aa7009f64531bfaf29fac8f1286b, user_id=361c9d3f558c4abf9a1c21095a30876f, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.190341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f18c0-f0f0-40a2-b852-c946750f1e73 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.196654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1092/2174] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:35.201419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.205318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1083/2175] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:35 2022] PUT /identity/v3/projects/87c6aa7009f64531bfaf29fac8f1286b/users/361c9d3f558c4abf9a1c21095a30876f/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:35.211461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8727f01-1d70-4c00-9dee-8d65074b43f5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.215536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.216497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.216778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.245243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] MFA Rules not processed for user `361c9d3f558c4abf9a1c21095a30876f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:35.246398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.271283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.271976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.272254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.272531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.273209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.274862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.285641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1093/2176] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:25:35.290066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.300361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f9e8b-1255-43ec-9b78-3b35d669463f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.305074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.305902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.306178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.306454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] PATH_INFO: `/v3/projects/8eddf8605a024cd1a172758a6552ae79/users/425d7830666b459da51f909f9338480d/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.315538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1084/2177] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:35.321530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eddf8605a024cd1a172758a6552ae79, user_id=425d7830666b459da51f909f9338480d, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.324245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aac3b-0e15-46e1-a1a5-2c6eb43ca6c9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.326542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.327932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.337990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1094/2178] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:35 2022] PUT /identity/v3/projects/8eddf8605a024cd1a172758a6552ae79/users/425d7830666b459da51f909f9338480d/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:35.348689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.349003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.349281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.379576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] MFA Rules not processed for user `425d7830666b459da51f909f9338480d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:35.380763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.384689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:35.387394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0242c75-3b1d-49cf-af8f-86386ae81d5e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.391559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:35.442632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.443242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.443519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.443785 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.448353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1095/2179] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:35.453171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.500459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.501881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.510829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1085/2180] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:35.513696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb4ce-b553-41cd-ba53-2fc9ca2c868e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.518569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.519176 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.519452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.519719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.520346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.522274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.530920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.541658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a2998-56c9-436d-9d17-dc35738d39ce service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.546897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.547516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.547795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.548066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] PATH_INFO: `/v3/users/361c9d3f558c4abf9a1c21095a30876f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.549343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=361c9d3f558c4abf9a1c21095a30876f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.550629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1096/2181] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/projects => generated 401 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:35.551436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.554637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.600429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361c9d3f558c4abf9a1c21095a30876f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:35.601130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361c9d3f558c4abf9a1c21095a30876f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:35.601755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361c9d3f558c4abf9a1c21095a30876f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:35.608589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361c9d3f558c4abf9a1c21095a30876f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:35.621498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514f261-2d4e-47bb-85ed-6060e7f6e89d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.621729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-31b0372a-b368-41d2-8823-7dd1c48da0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361c9d3f558c4abf9a1c21095a30876f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:35.626606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.627282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.627745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.628185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.628961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.630889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.634580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1086/2182] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:35 2022] DELETE /identity/v3/users/361c9d3f558c4abf9a1c21095a30876f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:35.639628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.639929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.640196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.656715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1097/2183] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:35.657819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.659450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.663598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:35.664803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.671832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381812c-9aa1-48a6-bd42-adaa132f1a94 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.690929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:35.696752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:35.738051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1087/2184] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:35.743420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.744031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.744315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.744587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.745264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.747601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.748532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.750595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:35.760813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1098/2185] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:25:35.761828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.771527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b99cb-bd2a-47fe-a235-596db7aafa7a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.776173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.776807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.777120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.777395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] PATH_INFO: `/v3/projects/7f04d78e589e4463bd76a413e58d8e1a/users/5a22bf37377e47d9b04efcb18eb80482/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.793656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f04d78e589e4463bd76a413e58d8e1a, user_id=5a22bf37377e47d9b04efcb18eb80482, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.797250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.808342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5880e0-88ca-400f-893a-3232302fdce3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.812519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1099/2186] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:35 2022] PUT /identity/v3/projects/7f04d78e589e4463bd76a413e58d8e1a/users/5a22bf37377e47d9b04efcb18eb80482/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:35.816196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.858560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.859322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.859872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.860011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.878456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bc0b3-136e-4b9f-a8e4-7dbb339520bb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.883100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.883696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.883976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.884251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.884893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.886507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.897001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1100/2187] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:25:35.901408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:35.912354 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad53a-d2a5-49ca-b6c0-73288417c4a0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:35.916908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:35.917570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.917838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.918116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] PATH_INFO: `/v3/projects/7f04d78e589e4463bd76a413e58d8e1a/users/5a22bf37377e47d9b04efcb18eb80482/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.920710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f04d78e589e4463bd76a413e58d8e1a, user_id=5a22bf37377e47d9b04efcb18eb80482, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.920824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:35.922202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353cbbd-3f41-4118-ac7d-44d44fa4d1bf service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.923684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:35.932079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1088/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:35.934277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1101/2189] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:35 2022] PUT /identity/v3/projects/7f04d78e589e4463bd76a413e58d8e1a/users/5a22bf37377e47d9b04efcb18eb80482/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:35.946636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:35.946924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:35.947222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:35.976799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] MFA Rules not processed for user `5a22bf37377e47d9b04efcb18eb80482`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:35.978416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.010064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:36.057616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1089/2190] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:36.069466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.071673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.158076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672a0f5-284e-44df-a9bf-2329fe64fe89 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.161731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.162330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.162606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.162874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.173685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.175076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.185984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1102/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:36 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:36.264997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.275767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159e0c-24c5-4411-a236-584464e079e3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.280904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.281539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.281809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.282070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.284765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.293682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.297464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.332120 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:36.374730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.376501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b54a-a2f5-4eae-9310-03f623ede4c4 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.385347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1090/2192] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:36.394416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c337df3-d9a4-49ed-8cc4-ff5b09bcd624 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.399337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.400046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.400366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.400706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] PATH_INFO: `/v3/projects/87c6aa7009f64531bfaf29fac8f1286b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.401991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c6aa7009f64531bfaf29fac8f1286b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.404819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.454014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c6aa7009f64531bfaf29fac8f1286b', 'request_id': 'req-d6ff8565-fc8d-40b5-a646-f90ad4115b69'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:36.459622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c6aa7009f64531bfaf29fac8f1286b', 'request_id': 'req-d6ff8565-fc8d-40b5-a646-f90ad4115b69'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:36.468313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c6aa7009f64531bfaf29fac8f1286b', 'request_id': 'req-d6ff8565-fc8d-40b5-a646-f90ad4115b69'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:36.479661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] The token cache is being invalidate because project 87c6aa7009f64531bfaf29fac8f1286b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:36.479952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:36.480596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d6ff8565-fc8d-40b5-a646-f90ad4115b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:36.482211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1103/2193] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:36 2022] DELETE /identity/v3/projects/87c6aa7009f64531bfaf29fac8f1286b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:36.487007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.487302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.487580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.507065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:36.508281 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.573584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1091/2194] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:36.582219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.582508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.582785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.604222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:36.605573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:36.615058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1104/2195] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:36.619288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.630723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f21006-b319-4bb3-a109-2dfdc136dbe2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.635758 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.636560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.636937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.637344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.638591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:36.639002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.640841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.656152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1092/2196] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:25:36.660813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.671732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902dbd2f-c5c8-48b9-af8e-a06acf5bfc2d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.676970 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.678346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.678694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.679095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.679928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.682708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.720527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1105/2197] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:36 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:36.724426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.735159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831f38d-ffab-480c-a653-8a404c4925ef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.781017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:36.790985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.791515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.791789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.792057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.792682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.795284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.815926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1093/2198] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:36 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:36.820046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.830149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924386-8639-4b02-a288-9f71c666a39c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.834667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.835250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.835522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.835789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.836421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.838055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.848134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1106/2199] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:36.851962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.862724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6125a2-0fda-43e8-8a94-ae66a69743f0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.867384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.867963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.868233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.868496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.869234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.870797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9639053-9efe-46b1-b54d-0b50c94c8650 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.881428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1094/2200] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:36.885358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:36.895270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd470-d5cd-40d4-b4cd-91c3f32884c5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:36.899777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:36.900410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.900683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.900978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] PATH_INFO: `/v3/projects/0439549da28c49ad8d2a1bc4eb81585e/users/4cb8e833405d42e6a0598a027fb33c26/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.915615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0439549da28c49ad8d2a1bc4eb81585e, user_id=4cb8e833405d42e6a0598a027fb33c26, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:36.918341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:36.932418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1107/2201] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:36 2022] PUT /identity/v3/projects/0439549da28c49ad8d2a1bc4eb81585e/users/4cb8e833405d42e6a0598a027fb33c26/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:36.943535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:36.943824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:36.944125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:36.976114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] MFA Rules not processed for user `4cb8e833405d42e6a0598a027fb33c26`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:36.977373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.041370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1095/2202] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:37.076845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:37.078606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.126030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:37.146238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b098-7b17-45e5-95b3-3300d0801486 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:37.146238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:37.146238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.146238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.146585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.156282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:37.157755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1ed9c-e79a-4088-93ea-f2659395d904 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:37.166546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1108/2203] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:37.182176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:37.245621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae9298-9594-44a6-b62d-b239e38df0dd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:37.250220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:37.250843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.251124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.251394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] PATH_INFO: `/v3/users/4cb8e833405d42e6a0598a027fb33c26` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.252548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb8e833405d42e6a0598a027fb33c26)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:37.254551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:37.304113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb8e833405d42e6a0598a027fb33c26'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.304664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb8e833405d42e6a0598a027fb33c26'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.305353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb8e833405d42e6a0598a027fb33c26'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.313205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb8e833405d42e6a0598a027fb33c26'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.317626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:37.328453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb8e833405d42e6a0598a027fb33c26'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.341367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1096/2204] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:37 2022] DELETE /identity/v3/users/4cb8e833405d42e6a0598a027fb33c26 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:37.345888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.346375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.346505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.365291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:37.366490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.428377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1109/2205] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:37.439351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:37.440746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.499227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641906-8994-499c-b217-1fd97ee5047d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:37.502966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:37.503561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.503831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.504096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.513983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:37.515407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02d41d-4532-4a13-8820-8c45388fc7c7 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:37.525174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1097/2206] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:25:37.788638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:37.790187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.808158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f8c0-87ce-415b-bdd7-e1212dd2c72d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:37.820559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:37.821229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.821508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.821776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] PATH_INFO: `/v3/projects/0439549da28c49ad8d2a1bc4eb81585e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.822910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0439549da28c49ad8d2a1bc4eb81585e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:37.824795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:37.873817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0439549da28c49ad8d2a1bc4eb81585e', 'request_id': 'req-5ae783e6-7f7e-40c6-a369-63e37df85d2a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.879441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0439549da28c49ad8d2a1bc4eb81585e', 'request_id': 'req-5ae783e6-7f7e-40c6-a369-63e37df85d2a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.888451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0439549da28c49ad8d2a1bc4eb81585e', 'request_id': 'req-5ae783e6-7f7e-40c6-a369-63e37df85d2a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.900102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] The token cache is being invalidate because project 0439549da28c49ad8d2a1bc4eb81585e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:37.900393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.901056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5ae783e6-7f7e-40c6-a369-63e37df85d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:37.902543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1110/2207] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:37 2022] DELETE /identity/v3/projects/0439549da28c49ad8d2a1bc4eb81585e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:37.908173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:37.908671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:37.908764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:37.931607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:37.933248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:37.996882 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1098/2208] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:38.004792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.005115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.005395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.031540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:38.033435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:38.041534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1111/2209] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:38.046104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.056402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5f15d-878d-485f-807d-7b4e80895bd0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.061963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.062583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.062859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.063128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.063768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:38.064067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.065712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.078525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1099/2210] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:38.082896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.092532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3173-b46d-4b53-9f40-78b41d35d21d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.097282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.097871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.098143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.098405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.099043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.100912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.129205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1112/2211] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:38 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:38.133461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.144301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2901c-e2ce-451d-b550-f1c37d5342ee admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.209221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.209833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.210106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.210371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.211004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.213046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.229619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:38.237412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1100/2212] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:38 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:38.241512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.252095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2315e7-5e4d-4682-a9a9-d868f9b0a13d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.256689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.257312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.257586 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.257852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.258487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.260075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.269910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1113/2213] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:38.273988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.284655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1430c9-84f7-465f-89bb-432adffda457 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.290437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.291264 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.292087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.292087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.292589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.294807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71247483-ee3c-4933-bc2a-5127d2dffa30 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.305470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1101/2214] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:38.310102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.320161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eae3ba-6cef-4a01-80df-c8e8cf734ad9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.324724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.325426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.325723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.325989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] PATH_INFO: `/v3/projects/d6475b33d8374da3a922f426843374f7/users/110487aeb8f74b878e6a665f3eaeb372/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.340405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6475b33d8374da3a922f426843374f7, user_id=110487aeb8f74b878e6a665f3eaeb372, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.343590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.357525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1114/2215] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:38 2022] PUT /identity/v3/projects/d6475b33d8374da3a922f426843374f7/users/110487aeb8f74b878e6a665f3eaeb372/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:38.367655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.368037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.368445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.399199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] MFA Rules not processed for user `110487aeb8f74b878e6a665f3eaeb372`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:38.400307 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:38.463992 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1102/2216] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:38.475611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:38.477294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:38.486986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:38.536184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe4020-5971-4966-8fa2-022953a796ee admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:38.540035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:38.540628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:38.540902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:38.541200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:38.549716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:38.551440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45312014-3fa7-4e41-b1cc-d2dbbdbdef8c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:38.560859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1115/2217] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:40.529389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:40.531015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:40.604107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050377f4-2e1a-4c71-af1d-e806c0c45853 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:40.608623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:40.609279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:40.609563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:40.609830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] PATH_INFO: `/v3/users/425d7830666b459da51f909f9338480d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:40.610935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425d7830666b459da51f909f9338480d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:40.612817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:40.661527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425d7830666b459da51f909f9338480d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:40.662313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425d7830666b459da51f909f9338480d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:40.662932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425d7830666b459da51f909f9338480d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:40.670284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425d7830666b459da51f909f9338480d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:40.684342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425d7830666b459da51f909f9338480d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:40.689404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:40.704797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1103/2218] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:40 2022] DELETE /identity/v3/users/425d7830666b459da51f909f9338480d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:40.709208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:40.709628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:40.711050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:40.733277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:40.735360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:40.808993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1116/2219] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:40.820339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:40.821795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:40.895492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a19f6-df49-4b54-9e22-e53df8901341 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:40.899232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:40.899960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:40.900110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:40.900387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:40.910557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:40.911977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:40.922389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1104/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:41.200286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:41.201865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:41.218531 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb82649-3016-4dbe-9fc5-c09cbed734f4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:41.223530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:41.224151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:41.224428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:41.224748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] PATH_INFO: `/v3/projects/8eddf8605a024cd1a172758a6552ae79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:41.225923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eddf8605a024cd1a172758a6552ae79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:41.227834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:41.274014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eddf8605a024cd1a172758a6552ae79', 'request_id': 'req-c9c14c22-7094-42b7-bb85-103876556682'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.279962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eddf8605a024cd1a172758a6552ae79', 'request_id': 'req-c9c14c22-7094-42b7-bb85-103876556682'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.287181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eddf8605a024cd1a172758a6552ae79', 'request_id': 'req-c9c14c22-7094-42b7-bb85-103876556682'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.298865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] The token cache is being invalidate because project 8eddf8605a024cd1a172758a6552ae79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:41.299147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.299837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.301423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1117/2221] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:41 2022] DELETE /identity/v3/projects/8eddf8605a024cd1a172758a6552ae79 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:41.305427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:41.315410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d68be-0461-4fff-9c34-1052a5ce0694 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:41.371242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:41.371921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:41.372220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:41.372553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] PATH_INFO: `/v3/users/5a22bf37377e47d9b04efcb18eb80482` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:41.373841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a22bf37377e47d9b04efcb18eb80482)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:41.375953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:41.421561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a22bf37377e47d9b04efcb18eb80482'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.422244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a22bf37377e47d9b04efcb18eb80482'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.422847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a22bf37377e47d9b04efcb18eb80482'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.429664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a22bf37377e47d9b04efcb18eb80482'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.442056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7801b7f5-bfbd-4e71-96d0-b5c2026172fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a22bf37377e47d9b04efcb18eb80482'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.454820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1105/2222] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:41 2022] DELETE /identity/v3/users/5a22bf37377e47d9b04efcb18eb80482 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:41.722482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:41.724028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:41.738545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14c22-7094-42b7-bb85-103876556682 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:41.775465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:41.803881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:41.804471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:41.804751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:41.805061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] PATH_INFO: `/v3/projects/7f04d78e589e4463bd76a413e58d8e1a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:41.806435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f04d78e589e4463bd76a413e58d8e1a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:41.808338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:41.854340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f04d78e589e4463bd76a413e58d8e1a', 'request_id': 'req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.859973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f04d78e589e4463bd76a413e58d8e1a', 'request_id': 'req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.868288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f04d78e589e4463bd76a413e58d8e1a', 'request_id': 'req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.880772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] The token cache is being invalidate because project 7f04d78e589e4463bd76a413e58d8e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:41.881101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.881785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6f579e1b-ee82-48ee-b1ee-2e0a56bacb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:41.883513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1118/2223] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:41 2022] DELETE /identity/v3/projects/7f04d78e589e4463bd76a413e58d8e1a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:41.889214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:41.889634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:41.889634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:41.909076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:41.910269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:41.922874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:41.971462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1106/2224] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:41.979097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:41.979407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:41.979679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:41.999859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:42.001085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:42.008864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1119/2225] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:42.013066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.023625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39690309-2041-4ac4-a904-a7115ea6ec24 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.028140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.028747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.029289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.029610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.030257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:42.030563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.032150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.043503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1107/2226] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:42.047663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.056992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee2f2e-5cfd-45f9-9958-c161f6fa445c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.062079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.062667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.062952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.063221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.063846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.065750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.098716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1120/2227] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:42.100001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.110791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e9308-829d-4735-9ca2-bce3256a2707 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.177824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.178427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.178700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.178967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.179601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.181786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.206193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1108/2228] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:42.210260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.221534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4d49-537c-4a0a-820a-1e2dae1866b8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.227756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.228497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.228901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.229199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.230513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.232517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.244730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1121/2229] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:42.246560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.256473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afabc9-e069-4ed4-b570-20c7b6850ec3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.261171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.261759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.262037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.262310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.262947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.265073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a02514-e251-49d5-9597-d6b3ed99e259 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.275400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1109/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:42.280028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.290014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011eb55-6b94-433f-8a17-0596256ef542 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.294560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.295231 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.295505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.295773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] PATH_INFO: `/v3/projects/dccae48efd9943aa9b27615ea64a3a43/users/a9a59c9b1eac49f8b6df9d6d71b06004/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.310498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccae48efd9943aa9b27615ea64a3a43, user_id=a9a59c9b1eac49f8b6df9d6d71b06004, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.313052 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.326104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1122/2231] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:42 2022] PUT /identity/v3/projects/dccae48efd9943aa9b27615ea64a3a43/users/a9a59c9b1eac49f8b6df9d6d71b06004/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:42.337734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.338025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.338305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.366597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] MFA Rules not processed for user `a9a59c9b1eac49f8b6df9d6d71b06004`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:42.367795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:42.433883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1110/2232] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:42.438381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.501787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf174b98-7fc3-4a6a-97da-7dbe6de9083b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.506456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.507059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.507340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.507608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.508258 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.510207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.515414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:42.540615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1123/2233] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:42.544851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.555348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7fbe-ebfd-4e58-80a0-d49173f569db None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.565354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:42.613543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.614146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.614423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.614698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.615336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.617275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.642258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1111/2234] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:25:42.646219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.656887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b3767-ab6d-4a0b-8b0b-9e80efb9c89b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.661516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.662105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.662378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.662645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.663274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.664877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.675212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1124/2235] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:42.679443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.689486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b861b-88e4-4253-a6f5-6f5c1eb01534 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.694093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.694742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.695020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.695290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] PATH_INFO: `/v3/projects/d51d96cf77274290a27915fdf1602601/users/bc541f338f2747039d2f8b0fa1b01e08/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.710270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51d96cf77274290a27915fdf1602601, user_id=bc541f338f2747039d2f8b0fa1b01e08, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.712883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.726164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1112/2236] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:42 2022] PUT /identity/v3/projects/d51d96cf77274290a27915fdf1602601/users/bc541f338f2747039d2f8b0fa1b01e08/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:42.730043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.740477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb12a7-e7f7-4e3d-860c-72fb67f951d6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.796076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.796669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.796964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.797237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.797865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.799433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d845f47-b26b-4218-9856-264bc1c26a8a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.809275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1125/2237] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:42.813513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.823794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df7f8c-ee7f-48f0-8253-101160589088 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:42.828445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:42.829165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.829491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.829926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] PATH_INFO: `/v3/projects/d51d96cf77274290a27915fdf1602601/users/bc541f338f2747039d2f8b0fa1b01e08/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.832671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51d96cf77274290a27915fdf1602601, user_id=bc541f338f2747039d2f8b0fa1b01e08, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:42.837084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:42.849528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1113/2238] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:42 2022] PUT /identity/v3/projects/d51d96cf77274290a27915fdf1602601/users/bc541f338f2747039d2f8b0fa1b01e08/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:42.860396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:42.860674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:42.860964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:42.888687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] MFA Rules not processed for user `bc541f338f2747039d2f8b0fa1b01e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:42.889978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:42.959111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1126/2239] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:42.971790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:42.973705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:43.036833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609c7a-f58f-4f85-8c89-dd1af34e541b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:43.040506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:43.041140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:43.041418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:43.041688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:43.050582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:43.051994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:43.061697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1114/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:43.144477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:43.154559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e325-e5b7-4ec7-9d23-1fe20124eabf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:43.158111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:43.158726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:43.159003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:43.159274 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:43.213256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:43.218859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:43.220791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d98f1-6183-4460-bcd0-06e07b4b3406 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:43.229842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1127/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:53.057506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=64,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:53.206395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:53.225661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:53.271800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb557317-20f9-4eaa-8351-ad571852b9a6 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:53.276332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:53.277069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:53.277344 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:53.277610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] PATH_INFO: `/v3/users/a9a59c9b1eac49f8b6df9d6d71b06004` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:53.278871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a59c9b1eac49f8b6df9d6d71b06004)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:53.281453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:53.330397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a59c9b1eac49f8b6df9d6d71b06004'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.331055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a59c9b1eac49f8b6df9d6d71b06004'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.331676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a59c9b1eac49f8b6df9d6d71b06004'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.338849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a59c9b1eac49f8b6df9d6d71b06004'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.351333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a59c9b1eac49f8b6df9d6d71b06004'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.365231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1115/2242] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:53 2022] DELETE /identity/v3/users/a9a59c9b1eac49f8b6df9d6d71b06004 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:53.370048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:53.370400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:53.370679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:53.390924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:53.392132 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:53.454774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1128/2243] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:53.464364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:53.465844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:53.521459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e4bd-0c71-4b80-97d9-11e7e425071f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:53.525057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:53.525653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:53.525928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:53.526199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:53.535942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:53.537390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:53.546869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1116/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:25:53.817462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:53.818926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:53.832418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427513d-ac27-43d8-a974-fb59985fbb8c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:53.837799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:53.838512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:53.838856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:53.839193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] PATH_INFO: `/v3/projects/dccae48efd9943aa9b27615ea64a3a43` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:53.840436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccae48efd9943aa9b27615ea64a3a43)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:53.842542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:53.896380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccae48efd9943aa9b27615ea64a3a43', 'request_id': 'req-25c83b9f-9df7-4930-a164-62bb85b841b0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.902405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccae48efd9943aa9b27615ea64a3a43', 'request_id': 'req-25c83b9f-9df7-4930-a164-62bb85b841b0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.910330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccae48efd9943aa9b27615ea64a3a43', 'request_id': 'req-25c83b9f-9df7-4930-a164-62bb85b841b0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.922391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] The token cache is being invalidate because project dccae48efd9943aa9b27615ea64a3a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:53.922776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.923489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:53.925079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1129/2245] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:53 2022] DELETE /identity/v3/projects/dccae48efd9943aa9b27615ea64a3a43 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:53.929170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:53.941874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b6d15-2a27-40cc-a497-c3cf05379962 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:53.969514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:54.021815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.022190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.022470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.022736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] PATH_INFO: `/v3/users/bc541f338f2747039d2f8b0fa1b01e08` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.023844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc541f338f2747039d2f8b0fa1b01e08)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.025980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.082864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc541f338f2747039d2f8b0fa1b01e08'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.084009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc541f338f2747039d2f8b0fa1b01e08'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.084830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc541f338f2747039d2f8b0fa1b01e08'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.092718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc541f338f2747039d2f8b0fa1b01e08'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.109050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9107a34f-da49-4dd4-9360-29f379d8ffdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc541f338f2747039d2f8b0fa1b01e08'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.129097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1117/2246] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:25:53 2022] DELETE /identity/v3/users/bc541f338f2747039d2f8b0fa1b01e08 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:54.400197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.401743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:54.417717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b9f-9df7-4930-a164-62bb85b841b0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.475832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.476457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.476732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.477033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] PATH_INFO: `/v3/projects/d51d96cf77274290a27915fdf1602601` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.478146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51d96cf77274290a27915fdf1602601)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.480044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.510588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:54.529101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51d96cf77274290a27915fdf1602601', 'request_id': 'req-8cfdbfaf-7270-419b-861f-13d9ada27296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.535169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51d96cf77274290a27915fdf1602601', 'request_id': 'req-8cfdbfaf-7270-419b-861f-13d9ada27296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.542284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51d96cf77274290a27915fdf1602601', 'request_id': 'req-8cfdbfaf-7270-419b-861f-13d9ada27296'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.554661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] The token cache is being invalidate because project d51d96cf77274290a27915fdf1602601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:25:54.555059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.555683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8cfdbfaf-7270-419b-861f-13d9ada27296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:25:54.557434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1130/2247] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:25:54 2022] DELETE /identity/v3/projects/d51d96cf77274290a27915fdf1602601 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:25:54.562240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.562527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.562801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.581775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:54.582947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:54.640243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1118/2248] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:25:54.647924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.648204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.648468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.668087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:54.669298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:54.677822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1131/2249] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:54.682942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.693505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695f279-3fab-4888-8106-77f838a1fabc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.698244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.698837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.699111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.699377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.700016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:25:54.700345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.702127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.713669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1119/2250] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:25:54.719111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.729158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf170275-b497-41b5-87d5-c3e35dbdca62 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.735380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.735991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.736272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.736557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.737239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.739151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.766376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1132/2251] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:25:54 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:54.770480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.780965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5d0ac-c2ec-4578-8934-0fe5b675f84b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.808343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:25:54.839176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.839783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.840057 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.840326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.840989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.843419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.865103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1120/2252] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:25:54 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:25:54.869467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.879624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdc671-1df4-4ef4-bad2-88553f697265 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.884649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.885365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.885640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.885907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.886544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.888112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.899716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1133/2253] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:54.906465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.916877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b211c-d708-466d-9cfc-eaaa2ca1125b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.921508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.922094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.922372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.922686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.923315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.924968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a743754-b4ac-4de0-8162-00dcf7ad9eac admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.936978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1121/2254] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:25:54.941591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:54.952694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cd1a8-4462-405f-870a-9d581447e450 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:54.957642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:54.958369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:54.958645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:54.958910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] PATH_INFO: `/v3/projects/90e2b7b61c1542a69a7114a543817fa1/users/bfe22ab0803c48ff85f87566df8c6b6f/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:54.973699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e2b7b61c1542a69a7114a543817fa1, user_id=bfe22ab0803c48ff85f87566df8c6b6f, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:54.976306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:54.988584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1134/2255] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:25:54 2022] PUT /identity/v3/projects/90e2b7b61c1542a69a7114a543817fa1/users/bfe22ab0803c48ff85f87566df8c6b6f/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:25:54.999736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:55.000021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:55.000294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:55.028156 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] MFA Rules not processed for user `bfe22ab0803c48ff85f87566df8c6b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:25:55.029413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:55.094827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1122/2256] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:25:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:25:55.109021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:25:55.111363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:25:55.171256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00088d3e-6bbd-4ee5-a925-76615a82e30e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:25:55.174866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:25:55.175467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:25:55.175744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:25:55.176038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:25:55.184587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:25:55.186091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95e806-81d3-4e18-812d-0ab7bcec40a0 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:25:55.195031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1135/2257] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:25:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:00.279448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:00.280973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:00.344363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d9c7-7f94-407e-9d26-60b3506ee968 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:00.349719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:00.350472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:00.350799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:00.351085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] PATH_INFO: `/v3/users/f280e9b875264544b92c1143086a6be9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:00.352358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f280e9b875264544b92c1143086a6be9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:00.354611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:00.394316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:00.401199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f280e9b875264544b92c1143086a6be9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.402369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f280e9b875264544b92c1143086a6be9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.403140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f280e9b875264544b92c1143086a6be9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.410524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f280e9b875264544b92c1143086a6be9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.422883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f280e9b875264544b92c1143086a6be9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.435821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1123/2258] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:00 2022] DELETE /identity/v3/users/f280e9b875264544b92c1143086a6be9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:00.441049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:00.441507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:00.441614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:00.460064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:00.461298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:00.508847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:00.522768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1136/2259] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:00.533350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:00.535044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:00.593627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9853-ea70-4d7b-a504-ce9bba552e0a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:00.597328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:00.597928 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:00.598202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:00.598471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:00.607280 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:00.609298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be03928-29d4-406c-b109-edb2f016fb3f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:00.618560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1124/2260] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:00.887659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:00.889636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:00.907553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238694a-cbd8-4044-be43-baea20ee123c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:00.912548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:00.913260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:00.913552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:00.913868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] PATH_INFO: `/v3/projects/3090754f4dd543518b950f21cef74587` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:00.915295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3090754f4dd543518b950f21cef74587)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:00.916887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:00.974466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3090754f4dd543518b950f21cef74587', 'request_id': 'req-12d4bd18-d623-407e-bc02-f3c33a4ccb25'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.979881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3090754f4dd543518b950f21cef74587', 'request_id': 'req-12d4bd18-d623-407e-bc02-f3c33a4ccb25'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.987164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3090754f4dd543518b950f21cef74587', 'request_id': 'req-12d4bd18-d623-407e-bc02-f3c33a4ccb25'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.998406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] The token cache is being invalidate because project 3090754f4dd543518b950f21cef74587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:00.998799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:00.999408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-12d4bd18-d623-407e-bc02-f3c33a4ccb25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:01.000863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1137/2261] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:00 2022] DELETE /identity/v3/projects/3090754f4dd543518b950f21cef74587 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:01.005663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.005949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.006221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.026114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:01.027316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:01.093350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1125/2262] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:01.101499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.101787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.102072 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.130547 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:01.131743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:01.140986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1138/2263] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:01.146216 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.156773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2a3e-3836-4f1d-b41d-b5c476ce4501 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.163697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.164325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.164602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.164872 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.165554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:26:01.165860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.167462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.185167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1126/2264] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:26:01.186149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.197785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343de058-d483-4848-b78c-d2eb15be0d8a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.204082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.204850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.205172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.205577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.206228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.208137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.246122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1139/2265] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:01 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:01.247333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.257289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85a912-65e5-4df4-8a5f-961ee56a32df admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.302961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:01.318528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.319127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.319424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.319695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.320323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.322397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.344815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1127/2266] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:01 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:01.348911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.359620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58a243-61a4-4efb-b8f8-da62015afd71 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.365144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.365873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.366194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.366470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.367252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.369413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.382912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1140/2267] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:26:01.386964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.397496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801554-df33-4c2c-a69c-1755c6778834 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.402003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.402588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.402858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.403126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.403752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.405513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b0d2a-77a6-4591-b73c-5215f446b334 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.415922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1128/2268] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:01.419835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.429939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963201-fb91-42d8-a357-0143ba7f6afb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.434885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.435523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.436300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.436758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] PATH_INFO: `/v3/projects/a3324b12312f43ebab7273b438ad9b65/users/06451ea3cd91471c9b09966516090509/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.451267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3324b12312f43ebab7273b438ad9b65, user_id=06451ea3cd91471c9b09966516090509, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.454804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.468572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1141/2269] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:01 2022] PUT /identity/v3/projects/a3324b12312f43ebab7273b438ad9b65/users/06451ea3cd91471c9b09966516090509/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:01.473620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.473901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.474168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.492406 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:01.493623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:01.553351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1129/2270] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:01.564840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:01.566600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:01.625756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f842e6-0267-4eea-b1c1-c9712e205d94 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:01.629398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:01.629994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:01.630262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:01.630526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:01.639734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:01.641305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4b02b-eb16-424d-8538-095fc02fee4d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:01.650352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1142/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:26:03.777144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:03.779072 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:03.793772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af985b27-d55b-4af1-a08a-4349192ea2a5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:03.799336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:03.799971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:03.800246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:03.800520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] PATH_INFO: `/v3/users/110487aeb8f74b878e6a665f3eaeb372` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:03.801661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=110487aeb8f74b878e6a665f3eaeb372)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:03.803567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:03.875337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110487aeb8f74b878e6a665f3eaeb372'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:03.876024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110487aeb8f74b878e6a665f3eaeb372'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:03.876723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110487aeb8f74b878e6a665f3eaeb372'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:03.884656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110487aeb8f74b878e6a665f3eaeb372'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:03.899412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110487aeb8f74b878e6a665f3eaeb372'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:03.919839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1130/2272] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:03 2022] DELETE /identity/v3/users/110487aeb8f74b878e6a665f3eaeb372 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:03.922900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:03.923877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:03.924305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:03.947136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:03.948629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:03.965248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:04.028918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1143/2273] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:04.056843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:04.062653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:04.115776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:04.146379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1f1f6-b6e3-4947-8c6e-ed4e7955be7a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:04.150950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:04.151615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:04.151888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:04.152157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:04.166315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:04.168984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1fb71-cf58-435b-9eee-f7f4a1761e24 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:04.182494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1131/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:04.604313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:04.606200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:04.630027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4406408-23a5-4598-9b24-e918113c20e5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:04.635733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:04.637200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:04.637962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:04.638331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] PATH_INFO: `/v3/projects/d6475b33d8374da3a922f426843374f7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:04.640715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6475b33d8374da3a922f426843374f7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:04.642791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:04.749055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6475b33d8374da3a922f426843374f7', 'request_id': 'req-2cfd418c-2ade-4836-bf81-238fababda2d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:04.760212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6475b33d8374da3a922f426843374f7', 'request_id': 'req-2cfd418c-2ade-4836-bf81-238fababda2d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:04.773999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6475b33d8374da3a922f426843374f7', 'request_id': 'req-2cfd418c-2ade-4836-bf81-238fababda2d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:04.793035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] The token cache is being invalidate because project d6475b33d8374da3a922f426843374f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:04.793365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:04.794157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2cfd418c-2ade-4836-bf81-238fababda2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:04.796753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1144/2275] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:04 2022] DELETE /identity/v3/projects/d6475b33d8374da3a922f426843374f7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:04.802334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:04.802865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:04.802865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:04.827696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:04.829018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:04.918741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1132/2276] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:04.925810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:04.926362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:04.926574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:04.963972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:04.966216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:04.982359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:04.988128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1145/2277] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:04.995622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4238e-bb7f-436c-83f5-839ce3270979 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.002414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.003036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.003310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.003575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.004297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:26:05.004603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.006883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.023944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1133/2278] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:26:05.028627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.039357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ee9d-3f8d-433a-ba8e-ad624b4772f3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.046433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.047051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.047323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.047657 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.048587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.051071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.095647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1146/2279] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:05 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:05.099289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.111176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2015b-9c18-4be4-94c8-f1bed7e65f27 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.186779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.187765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.188110 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.188445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.189201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.191322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.224770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1134/2280] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:05 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:05.226395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.240381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a9ed2-9acb-4476-9de3-fc2ff5d3473f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.246360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.247057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.247594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.247933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.249311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.251227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.262810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1147/2281] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:26:05.267454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.284516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd00b4-aac9-48f4-87b5-08f3da62a7d9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.304630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.305284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.305557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.305824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.306902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.308822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840edf99-a37a-4976-a71c-2f56def68a8b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.324991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1135/2282] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:05.329591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.340496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d57b-61fc-439b-b62a-ec1730c8f3c2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.346653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.347787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.347787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.348043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] PATH_INFO: `/v3/projects/50299ae9b0cc42fda5f8a0f0e1c1b9bf/users/550b583ec63c46c4b2cf50ec4a0f6adc/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.364417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50299ae9b0cc42fda5f8a0f0e1c1b9bf, user_id=550b583ec63c46c4b2cf50ec4a0f6adc, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.367095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.384417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1148/2283] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:05 2022] PUT /identity/v3/projects/50299ae9b0cc42fda5f8a0f0e1c1b9bf/users/550b583ec63c46c4b2cf50ec4a0f6adc/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:05.395674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.395854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.396030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.427787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] MFA Rules not processed for user `550b583ec63c46c4b2cf50ec4a0f6adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:05.429363 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:05.484618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:05.542778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1136/2284] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:05.547094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.618647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:05.653844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f03ce-6f34-4b3f-91a7-25367095dbd3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.660557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.662632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.662910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.663174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.664206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.666269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.704817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1149/2285] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:05 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:05.708937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.731288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367699c-448c-4538-9391-dcdfd1ad4d41 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.811510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.812835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.813397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.814095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.814753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.817446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.845175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1137/2286] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:05 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:05.848896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.859790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa51-a471-4fe0-acb6-49d2d113847f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.864856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.865561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.865950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.866211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.866894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.868523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.884716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1150/2287] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:05.888451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.899860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6670-2f05-4dd3-bfcb-5965c097d665 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.906298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.906914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.907191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.907464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.908114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.909772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57a8ba-a6b4-4112-bc69-e488e07eb5ff admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.921192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1138/2288] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:05.925197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:05.938535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b966-2cb9-470f-80e8-71bbe668dfb6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:05.944323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:05.945211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:05.945551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:05.945884 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] PATH_INFO: `/v3/projects/d2c6b7137e17488a9c795383fb827dee/users/044d2c15c42b4914859c816458828595/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:05.963620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c6b7137e17488a9c795383fb827dee, user_id=044d2c15c42b4914859c816458828595, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:05.966582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:05.990821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1151/2289] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:05 2022] PUT /identity/v3/projects/d2c6b7137e17488a9c795383fb827dee/users/044d2c15c42b4914859c816458828595/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:06.002555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.003022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.003566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.035063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] MFA Rules not processed for user `044d2c15c42b4914859c816458828595`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:06.036291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:06.120767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1139/2290] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:06.128357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.258401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba37a2-b171-4e7e-9191-9f92351d29a0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.266384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.267104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.267448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.267803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.268527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.271683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.304084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1152/2291] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:06.308428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.322364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae34bba-336c-4b5f-b1a9-8f56cdf56edf None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.392038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:06.414469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.415544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.415887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.416166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.417694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.419635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eced0c7-e4db-4a5d-8d9c-bc27027df390 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.481544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1140/2292] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:06.484953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.507528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c5f3e-b5f0-4a15-83b0-029cac7989f6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.523554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.523554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.523554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.523554 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.523920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.523920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.524977 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.525276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.525553 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.534288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1153/2293] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:26:06.538649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.549563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe7dc-edf2-4018-906a-3d9b509889a3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.554573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.555423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.555705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.556234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] PATH_INFO: `/v3/projects/4050a97afc5b4d61ac47d44c09dbc412/users/222a9f2e16a349568ce933f22d3e8d83/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.557753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] MFA Rules not processed for user `06451ea3cd91471c9b09966516090509`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:06.559070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:06.576227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4050a97afc5b4d61ac47d44c09dbc412, user_id=222a9f2e16a349568ce933f22d3e8d83, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.580112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.597813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1154/2294] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:06 2022] PUT /identity/v3/projects/4050a97afc5b4d61ac47d44c09dbc412/users/222a9f2e16a349568ce933f22d3e8d83/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:06.602625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.647585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1141/2295] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:06.651735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.653429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:06.658768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:06.672442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fcb9a-f43b-49a3-87e9-83d24db62e40 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.705226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9b50-7739-467f-9ff6-4f1df12fb91f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.710140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.710762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.711041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.711426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.711969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.713661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.725217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1155/2296] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:06.734165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.747952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.749718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.749996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.750265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.750918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.751170 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831416-41c4-46d0-89de-280beab01cbd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.752837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.756311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.757143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.757603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.758045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] PATH_INFO: `/v3/projects/4050a97afc5b4d61ac47d44c09dbc412/users/222a9f2e16a349568ce933f22d3e8d83/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.760469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4050a97afc5b4d61ac47d44c09dbc412, user_id=222a9f2e16a349568ce933f22d3e8d83, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.763287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52184f7e-1ff4-4eef-a064-d95ff6e2c39d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.776646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1156/2297] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:06 2022] PUT /identity/v3/projects/4050a97afc5b4d61ac47d44c09dbc412/users/222a9f2e16a349568ce933f22d3e8d83/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:06.788213 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.788779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.789319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.797149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1142/2298] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:06.801046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.834029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] MFA Rules not processed for user `222a9f2e16a349568ce933f22d3e8d83`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:06.835476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:06.888547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:06.908903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b454cd3-7e9a-42a5-a395-032394b06d6a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:06.917176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:06.917814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:06.918094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:06.918361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:06.919468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:06.922939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:06.928754 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1157/2299] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:06.942173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.943734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:06.957796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1143/2300] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:06 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:06.962626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:06.984402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2945c-5c13-497b-be16-c613ffa5509a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:07.007003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:07.007637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:07.007897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:07.008167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:07.009233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:07.010846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:07.028428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1144/2301] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:07.033793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:07.045988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd3745-3f01-46a5-8fbd-94cf4d7de4cb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:07.058814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:07.059405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:07.059681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:07.059953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] PATH_INFO: `/v3/projects/45f7438a4c6f4d28a9776154a47aa1de/users/f552039a4b76474991367034bb909723/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:07.072990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd962cc-011b-4995-bc48-e7df742bc70e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:07.080113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:07.082823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f7438a4c6f4d28a9776154a47aa1de, user_id=f552039a4b76474991367034bb909723, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:07.082978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:07.082978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:07.084397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:07.084851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:07.104560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1145/2302] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:07 2022] PUT /identity/v3/projects/45f7438a4c6f4d28a9776154a47aa1de/users/f552039a4b76474991367034bb909723/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:07.110154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:07.169729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:07.171332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:07.175355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:07.183259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1158/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:07.187656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8492-25ee-47ad-98f4-8a2c75c9e6a5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:07.192662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:07.193313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:07.193593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:07.193865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:07.194534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:07.196258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:07.207631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1146/2304] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:07.211818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:07.223981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd2988-5f83-40b8-a0d1-1f1fd82bdc97 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:07.229424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:07.230869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:07.231155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:07.231522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] PATH_INFO: `/v3/projects/45f7438a4c6f4d28a9776154a47aa1de/users/f552039a4b76474991367034bb909723/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:07.234347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f7438a4c6f4d28a9776154a47aa1de, user_id=f552039a4b76474991367034bb909723, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:07.237435 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:07.251765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1159/2305] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:07 2022] PUT /identity/v3/projects/45f7438a4c6f4d28a9776154a47aa1de/users/f552039a4b76474991367034bb909723/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:08.952534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:09.056738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bce970-4430-48e8-bdcb-928f815e33bb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:09.139528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:09.140845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:09.140845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:09.145629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:09.159450 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:09.259540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:09.261251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fae015-ccf0-407c-88ff-1a083920e321 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:09.287256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1147/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:10.159594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:10.181372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb552f5-8b18-45da-ba69-1d8d5c2f6bf1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:10.185681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:10.187141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:10.190809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:10.191131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:10.278193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:10.281553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:10.306509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1160/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:13.136271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b76c-4ae3-42a4-93d7-bbe9aaa6f5e4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:13.136788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b76c-4ae3-42a4-93d7-bbe9aaa6f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:13.136919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b76c-4ae3-42a4-93d7-bbe9aaa6f5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:13.172920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8474b76c-4ae3-42a4-93d7-bbe9aaa6f5e4 None None] MFA Rules not processed for user `f552039a4b76474991367034bb909723`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:13.174397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8474b76c-4ae3-42a4-93d7-bbe9aaa6f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:13.260197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1148/2308] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:13.272453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:13.275048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:13.338470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9678f-e6f2-46b7-9362-c87c470418d5 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:13.343043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:13.343780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:13.344386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:13.345259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:13.428916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:13.430420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:13.444570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1161/2309] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:13.615287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:13.615287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:13.615287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:13.656096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:13.657603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:13.722669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1149/2310] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:13.778464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:13.780583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:13.819640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:13.859519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442974-dd81-43dd-b47c-3788a9918403 service glance] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:13.864960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:13.865772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:13.866121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:13.866457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:13.875590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:13.877198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ebecc-4918-4d7f-a6d2-de8abcbdd2bc service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:13.886712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1162/2311] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:26:14.584297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:14.711843 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:14.773267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277394-6600-4805-bf79-e3af15ebbec9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:14.791840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:14.792676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:14.793041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:14.793282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] PATH_INFO: `/v3/users/550b583ec63c46c4b2cf50ec4a0f6adc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:14.794438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550b583ec63c46c4b2cf50ec4a0f6adc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:14.796908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:14.892629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550b583ec63c46c4b2cf50ec4a0f6adc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:14.893248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550b583ec63c46c4b2cf50ec4a0f6adc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:14.894033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550b583ec63c46c4b2cf50ec4a0f6adc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:14.919362 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550b583ec63c46c4b2cf50ec4a0f6adc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:14.941779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550b583ec63c46c4b2cf50ec4a0f6adc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:14.964120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1150/2312] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:14 2022] DELETE /identity/v3/users/550b583ec63c46c4b2cf50ec4a0f6adc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:14.969077 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:14.969915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:14.970361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:14.995259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:14.996796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:15.099769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1163/2313] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:15.114714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:15.116021 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:15.226226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726359f-994d-4723-b24f-296d3f846729 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:15.246394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:15.247335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:15.247607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:15.248044 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:15.264013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:15.267891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:15.305134 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1151/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:15.821660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:15.823502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:15.836328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07de71e-9838-44f2-9891-b47b4c9eae9c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:15.845611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:15.846262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:15.846545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:15.846819 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] PATH_INFO: `/v3/projects/50299ae9b0cc42fda5f8a0f0e1c1b9bf` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:15.848116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50299ae9b0cc42fda5f8a0f0e1c1b9bf)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:15.850025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:15.910845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50299ae9b0cc42fda5f8a0f0e1c1b9bf', 'request_id': 'req-2a575d61-c98b-41e0-b90a-76d282af818c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:15.919253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50299ae9b0cc42fda5f8a0f0e1c1b9bf', 'request_id': 'req-2a575d61-c98b-41e0-b90a-76d282af818c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:15.929063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50299ae9b0cc42fda5f8a0f0e1c1b9bf', 'request_id': 'req-2a575d61-c98b-41e0-b90a-76d282af818c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:15.944063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] The token cache is being invalidate because project 50299ae9b0cc42fda5f8a0f0e1c1b9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:15.944335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:15.945019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:15.946871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1164/2315] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:15 2022] DELETE /identity/v3/projects/50299ae9b0cc42fda5f8a0f0e1c1b9bf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:15.950650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:15.961981 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835de9-6838-4bfa-97ac-53a34d66ffbb service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:16.029146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:16.029146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:16.029146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:16.029146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] PATH_INFO: `/v3/users/044d2c15c42b4914859c816458828595` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:16.032913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044d2c15c42b4914859c816458828595)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:16.034937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:16.037739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:16.108536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044d2c15c42b4914859c816458828595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.109273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044d2c15c42b4914859c816458828595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.109870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044d2c15c42b4914859c816458828595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.117419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044d2c15c42b4914859c816458828595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.134652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044d2c15c42b4914859c816458828595'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.160566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1152/2316] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:15 2022] DELETE /identity/v3/users/044d2c15c42b4914859c816458828595 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:16.510133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:16.512360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:16.539990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575d61-c98b-41e0-b90a-76d282af818c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:16.587682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:16.609676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:16.610432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:16.610764 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:16.611106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] PATH_INFO: `/v3/projects/d2c6b7137e17488a9c795383fb827dee` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:16.612328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c6b7137e17488a9c795383fb827dee)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:16.614426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:16.699315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c6b7137e17488a9c795383fb827dee', 'request_id': 'req-725a109d-b1e6-4b42-a140-63fb487bd285'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.710272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c6b7137e17488a9c795383fb827dee', 'request_id': 'req-725a109d-b1e6-4b42-a140-63fb487bd285'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.739945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c6b7137e17488a9c795383fb827dee', 'request_id': 'req-725a109d-b1e6-4b42-a140-63fb487bd285'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.766140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] The token cache is being invalidate because project d2c6b7137e17488a9c795383fb827dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:16.766475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.767096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.768923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1165/2317] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:16 2022] DELETE /identity/v3/projects/d2c6b7137e17488a9c795383fb827dee => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:16.773922 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:16.776597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:16.791291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31573c0-ed50-46c8-a768-f26200030106 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:16.898246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:16.899221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:16.899792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:16.900314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] PATH_INFO: `/v3/users/222a9f2e16a349568ce933f22d3e8d83` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:16.902138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222a9f2e16a349568ce933f22d3e8d83)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:16.904880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:16.981097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222a9f2e16a349568ce933f22d3e8d83'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.983309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222a9f2e16a349568ce933f22d3e8d83'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:16.984205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222a9f2e16a349568ce933f22d3e8d83'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.003700 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222a9f2e16a349568ce933f22d3e8d83'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.025272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-16b10bf0-a43b-4c89-9774-17cb842211a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222a9f2e16a349568ce933f22d3e8d83'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.040257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:17.046936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1153/2318] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:16 2022] DELETE /identity/v3/users/222a9f2e16a349568ce933f22d3e8d83 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:17.421520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:17.423146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:17.447398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a109d-b1e6-4b42-a140-63fb487bd285 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:17.580111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:17.580784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:17.581055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:17.581234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] PATH_INFO: `/v3/projects/4050a97afc5b4d61ac47d44c09dbc412` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:17.583050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4050a97afc5b4d61ac47d44c09dbc412)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:17.584479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:17.676048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4050a97afc5b4d61ac47d44c09dbc412', 'request_id': 'req-fa966637-6406-436f-8cfd-311613c1f860'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.682314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4050a97afc5b4d61ac47d44c09dbc412', 'request_id': 'req-fa966637-6406-436f-8cfd-311613c1f860'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.695118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4050a97afc5b4d61ac47d44c09dbc412', 'request_id': 'req-fa966637-6406-436f-8cfd-311613c1f860'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.709337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] The token cache is being invalidate because project 4050a97afc5b4d61ac47d44c09dbc412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:17.709816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.710536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fa966637-6406-436f-8cfd-311613c1f860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:17.718100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1166/2319] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:17 2022] DELETE /identity/v3/projects/4050a97afc5b4d61ac47d44c09dbc412 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:17.720155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:17.720559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:17.720840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:17.743242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:17.744643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:17.824760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1154/2320] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:17.832773 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:17.833023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:17.833290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:17.852691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:17.863202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:17.864456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:17.878195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1167/2321] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:17.879606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:17.890961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32684025-63bb-409b-b551-84ce9bf041d7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:17.896118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:17.897089 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:17.899965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:17.900680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:17.901641 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:26:17.902184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:17.904062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:17.920969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1155/2322] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:26:17.926529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:17.937266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69fff-f07c-4ef4-ab82-f0e2bce59e99 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:17.945241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:17.946193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:17.948502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:17.949074 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:17.949939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:17.952812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:17.997931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1168/2323] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:17 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:18.002444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:18.030940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6b2f9-bf7d-4f3f-b414-b549bb21db57 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:18.143389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:18.144175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.144452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.144719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.145401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:18.157597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:18.204065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1156/2324] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:18 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:18.207996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:18.245678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0d8b1-67e3-40e6-b6be-bb052c899775 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:18.258233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:18.259044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.259326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.259624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.260277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:18.261943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:18.298782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:18.304185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1169/2325] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:26:18.309576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774820ed-c7e0-429b-8895-f16645ff2c13 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:18.318608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:18.319459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.320296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.320642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.321430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:18.323211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020f02c-e05e-407c-9dd3-12ae857adf74 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:18.338071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1157/2326] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:26:18.342350 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:18.363345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478046e3-8712-430f-817d-c22016f7b377 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:18.371255 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:18.371862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.372197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.372400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] PATH_INFO: `/v3/projects/3eaefceee20f4459bb33eae9b7a6d375/users/5a5ce54f49ce4cbab6bf38142b4e69f1/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.397070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaefceee20f4459bb33eae9b7a6d375, user_id=5a5ce54f49ce4cbab6bf38142b4e69f1, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:18.400977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:18.419115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1170/2327] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:18 2022] PUT /identity/v3/projects/3eaefceee20f4459bb33eae9b7a6d375/users/5a5ce54f49ce4cbab6bf38142b4e69f1/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:26:18.432881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66174e1-d8af-4abe-8d79-f6aa47a84a1f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.433184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66174e1-d8af-4abe-8d79-f6aa47a84a1f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.433457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66174e1-d8af-4abe-8d79-f6aa47a84a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.468161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d66174e1-d8af-4abe-8d79-f6aa47a84a1f None None] MFA Rules not processed for user `5a5ce54f49ce4cbab6bf38142b4e69f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:18.469475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d66174e1-d8af-4abe-8d79-f6aa47a84a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:18.557813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:18.571342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1158/2328] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:18.583684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:18.586832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:18.661986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643e0c-8f97-4665-9b46-7357b8ad7c72 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:18.665794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:18.671536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:18.671832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:18.672089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:18.682648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:18.684086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3452f7-3d60-49dc-85d8-3c1142387511 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:18.695953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1171/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:24.792268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:24.792854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:24.792854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:24.812985 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] MFA Rules not processed for user `06451ea3cd91471c9b09966516090509`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:24.814193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:24.870290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1159/2330] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:28.691136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:30.972341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:30.972988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:30.973210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:31.014732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:31.015171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:31.101265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1172/2331] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:26:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:31.111381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:31.113668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:31.249477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163901a-98c2-4909-ab9e-8100e743493d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:31.257269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:31.259537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:31.259908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:31.260435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:31.272775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:31.275320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a50ed-7eb5-4382-8b39-29fdaad90fa7 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:31.288163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1160/2332] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:26:36.090719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:36.092989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:36.249121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf37ff4-ca58-4b69-a5cb-a6b8f45f57d4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:36.249121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:36.249121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:36.249121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:36.249828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] PATH_INFO: `/v3/users/bfe22ab0803c48ff85f87566df8c6b6f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:36.250130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe22ab0803c48ff85f87566df8c6b6f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:36.257480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:36.316261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe22ab0803c48ff85f87566df8c6b6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:36.317004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe22ab0803c48ff85f87566df8c6b6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:36.326689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe22ab0803c48ff85f87566df8c6b6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:36.327442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe22ab0803c48ff85f87566df8c6b6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:36.342186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe22ab0803c48ff85f87566df8c6b6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:36.358944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1173/2333] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:36 2022] DELETE /identity/v3/users/bfe22ab0803c48ff85f87566df8c6b6f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:36.363522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:36.364068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:36.364206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:36.388347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:36.390628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:36.466105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1161/2334] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:36.476123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:36.477579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:36.520113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:36.558820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b9e0c-fb28-45df-9987-fadd908d1614 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:36.563880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:36.570437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:36.572366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:36.572930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:36.591144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:36.592788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df37d8-08a8-4109-be10-a3df40464e7b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:36.605717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1174/2335] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:37.110666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.112402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:37.135237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b344b-e555-4ef0-8427-f0d9b1c01f8e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.140901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.141628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.141898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.142166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] PATH_INFO: `/v3/projects/90e2b7b61c1542a69a7114a543817fa1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.143399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e2b7b61c1542a69a7114a543817fa1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.145370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.189234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:37.208006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e2b7b61c1542a69a7114a543817fa1', 'request_id': 'req-223c0ee2-08b1-4575-b6c6-e2c2740702c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:37.213862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e2b7b61c1542a69a7114a543817fa1', 'request_id': 'req-223c0ee2-08b1-4575-b6c6-e2c2740702c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:37.224100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e2b7b61c1542a69a7114a543817fa1', 'request_id': 'req-223c0ee2-08b1-4575-b6c6-e2c2740702c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:37.237650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] The token cache is being invalidate because project 90e2b7b61c1542a69a7114a543817fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:37.238279 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:37.238957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-223c0ee2-08b1-4575-b6c6-e2c2740702c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:37.240836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1162/2336] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:37 2022] DELETE /identity/v3/projects/90e2b7b61c1542a69a7114a543817fa1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:37.245682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.246220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.246475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.269757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:37.271017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:37.344320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1175/2337] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:37.352155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.352375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.352652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.376760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:37.379247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:37.408624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1163/2338] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:37.412524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.427235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92e544-03eb-4fd2-aa74-4bee2e4a3ca7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.433197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.433828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.434546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.434825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.436364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:26:37.436685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.439254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.476346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1176/2339] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:26:37.484962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.498246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348f008-5312-43bc-a769-1a9a00a4d708 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.505674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.506554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.507068 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.507403 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.508131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.510153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.541402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1164/2340] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:26:37 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:26:37.546795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.557578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1e9e-4023-4ae1-adb8-72bdd751489b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.616330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.616905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.617226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.617466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.618210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.620717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.647368 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1177/2341] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:26:37 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:26:37.651731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.663462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a180c1-feb4-4ce1-8e5b-3109556fd8f9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.668226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.668897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.669276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.670568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.670990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.672623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.687630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1165/2342] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:37.696319 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.707047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d0ca-08ad-4707-a13c-ab5ea91564be admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.711821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.712470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.712743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.713061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.713688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.715288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655d656-096e-4652-8778-1c7b723635d5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.727045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1178/2343] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:26:37.731062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.742480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132aacb-9219-460e-9a52-e31f47578081 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:37.748696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:37.749842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.750129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.750514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] PATH_INFO: `/v3/projects/54f4a6a2f9fe4b07b9cf74d700d36c3a/users/5a5201f196c34c10ab30840eca1ddced/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.768322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f4a6a2f9fe4b07b9cf74d700d36c3a, user_id=5a5201f196c34c10ab30840eca1ddced, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:37.771179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:37.785834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1166/2344] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:26:37 2022] PUT /identity/v3/projects/54f4a6a2f9fe4b07b9cf74d700d36c3a/users/5a5201f196c34c10ab30840eca1ddced/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:37.798131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:37.798423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:37.798700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:37.830658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] MFA Rules not processed for user `5a5201f196c34c10ab30840eca1ddced`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:37.831886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:37.852821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:37.912358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1179/2345] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:37.926480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:37.929605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:38.005802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499505c3-2157-4cd3-a532-a7c79d82b4bf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:38.009940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:38.010568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:38.010845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:38.011114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:38.022771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:38.025238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49bafe-1d26-445c-95a0-67e176d721bc service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:38.043999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1167/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:40.855931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:40.858005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:40.927336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0d6ab-b86d-46cc-82fd-07b99b1e3ac8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:40.931954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:40.932582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:40.932858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:40.933180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] PATH_INFO: `/v3/users/5a5ce54f49ce4cbab6bf38142b4e69f1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:40.934302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5ce54f49ce4cbab6bf38142b4e69f1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:40.936508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:40.994374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5ce54f49ce4cbab6bf38142b4e69f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:40.994940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5ce54f49ce4cbab6bf38142b4e69f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.001871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5ce54f49ce4cbab6bf38142b4e69f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.002509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5ce54f49ce4cbab6bf38142b4e69f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.020621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5ce54f49ce4cbab6bf38142b4e69f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.039620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1180/2347] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:26:40 2022] DELETE /identity/v3/users/5a5ce54f49ce4cbab6bf38142b4e69f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:41.041302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:41.041868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:41.041868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:41.070819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:41.072126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:41.137591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:41.171698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1168/2348] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:26:41.171926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:41.173404 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:41.232001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:41.252707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e520a-9f50-4a8a-863c-1911a04c65fa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:41.256280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:41.256900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:41.257206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:41.257580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:41.268363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:41.269839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e4ff2-a00d-4be8-ba64-06c06a4eba9f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:41.280476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1181/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:26:41.582647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:41.584658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:41.600055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c093d6-0537-439d-91f8-5107923e511b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:41.605657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:41.606385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:41.606739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:41.607099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] PATH_INFO: `/v3/projects/3eaefceee20f4459bb33eae9b7a6d375` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:41.608839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eaefceee20f4459bb33eae9b7a6d375)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:41.611329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:41.691912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eaefceee20f4459bb33eae9b7a6d375', 'request_id': 'req-e2f06e81-82e0-4071-ab08-21dad0560873'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.701769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eaefceee20f4459bb33eae9b7a6d375', 'request_id': 'req-e2f06e81-82e0-4071-ab08-21dad0560873'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.725250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eaefceee20f4459bb33eae9b7a6d375', 'request_id': 'req-e2f06e81-82e0-4071-ab08-21dad0560873'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.739062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] The token cache is being invalidate because project 3eaefceee20f4459bb33eae9b7a6d375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:26:41.739352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.740256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:26:41.741870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1169/2350] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:26:41 2022] DELETE /identity/v3/projects/3eaefceee20f4459bb33eae9b7a6d375 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:26:51.276219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:51.735814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:26:56.501130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fbcff-bb20-4ae8-9d75-45cd97faeae8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:56.502292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fbcff-bb20-4ae8-9d75-45cd97faeae8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:56.502650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fbcff-bb20-4ae8-9d75-45cd97faeae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:56.522464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f11fbcff-bb20-4ae8-9d75-45cd97faeae8 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:56.523887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f11fbcff-bb20-4ae8-9d75-45cd97faeae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:56.584303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1182/2351] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:26:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:26:56.599258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:26:56.600711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:56.697311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f06e81-82e0-4071-ab08-21dad0560873 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:26:56.702828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:56.709460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:56.709735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:56.719574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:26:56.722131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:26:56.722522 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:26:56.722897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:26:56.732650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:26:56.734815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:26:56.744925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1170/2352] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:26:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:26:56.746064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] MFA Rules not processed for user `d286b1e0afe9413291e0e2503b4c204a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:26:56.747284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:26:56.841100 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1183/2353] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:26:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:06.740578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=32 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:06.820559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:09.253776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:09.255245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:09.325302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e426fc-5533-4174-90ce-51a302176988 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:09.330239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:09.331356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:09.331627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:09.332005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:09.333613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:09.345238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:09.347653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d317c0-73d5-4c6d-89c8-934b559184a9 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:09.357629 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1171/2354] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:19.353593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=32 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:19.421182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:19.423318 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:19.499515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a221f43-c9f4-4456-9935-59ddf8780271 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:19.505058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:19.505762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:19.506113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:19.506451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:19.508316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:19.593795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:19.596362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e9188-4178-4497-89fe-b1c40f4b30bf service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:19.609455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1184/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:19.790269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:19.790645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:19.790848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:19.792185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1172/2356] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:27:19.799273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:19.799559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:19.800005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:19.822603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:19.823835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:19.892284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1185/2357] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:19.904382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:19.906161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:19.968991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6647d-b931-4fae-9bde-050ad8393869 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:19.974425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:19.975213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:19.975684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:19.976169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:19.989871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:19.991423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:19.994389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:19.995833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7ff1-e1b7-4382-8e66-1705302c366d service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:20.009989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1173/2358] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:20.061544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:20.086392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150379-8551-49ea-8d2e-28bcf4be7cb1 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.086724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b384a-59a3-4797-9fbc-0adc9ccfcf01 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:20.086830 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150379-8551-49ea-8d2e-28bcf4be7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.086939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150379-8551-49ea-8d2e-28bcf4be7cb1 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.088270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1174/2359] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:27:20.090960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:20.091582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.091983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.092117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] PATH_INFO: `/v3/users/5a5201f196c34c10ab30840eca1ddced` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.093069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.093289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5201f196c34c10ab30840eca1ddced)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:20.093425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.094128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.095203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:20.116162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:20.117550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:20.131180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1175/2360] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:27:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:27:20.144106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:20.155755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023b43-dfe8-4e36-bb17-3c538e03dbb0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:20.157436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5201f196c34c10ab30840eca1ddced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:20.158469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5201f196c34c10ab30840eca1ddced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:20.162591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:20.163313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.163817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.164224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.169559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5201f196c34c10ab30840eca1ddced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:20.170465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5201f196c34c10ab30840eca1ddced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:20.173492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:20.175111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2624-aa56-439c-a6f5-a65d277e69b8 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:20.187224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c6700e84-9bb5-4313-9b89-080f31d47f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5201f196c34c10ab30840eca1ddced'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:20.190004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1176/2361] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:20.218904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1186/2362] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:27:19 2022] DELETE /identity/v3/users/5a5201f196c34c10ab30840eca1ddced => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:27:20.223020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.223326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.223596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.251811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:20.253116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:20.293980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d167b-a864-4944-83e5-11c01d527fb0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.294263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d167b-a864-4944-83e5-11c01d527fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.294537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d167b-a864-4944-83e5-11c01d527fb0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.296031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1187/2363] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:27:20.301290 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.301572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.302590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.327253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:20.328508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:20.330814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1177/2364] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:27:20.342588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:20.344387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:20.410665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0d8a-2621-495b-84c6-b86a271804b0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:20.412975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1188/2365] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:27:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:27:20.416442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:20.416728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.417038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.417285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.424263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:20.429537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:20.431486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:20.433861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f85e17-9669-4180-a742-839d3bad35e2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:20.438334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:20.438940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:20.439214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:20.439484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:20.442290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1178/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:20.450653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:20.452092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e4268-e58a-4038-b271-00de8b296d95 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:20.464239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1189/2367] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:27:20.978409 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:20.980217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:20.996399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f28bfc-6190-4c57-8514-07a9539d946b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.001402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.002132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.002474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.002808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] PATH_INFO: `/v3/projects/54f4a6a2f9fe4b07b9cf74d700d36c3a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.004789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f4a6a2f9fe4b07b9cf74d700d36c3a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.006927 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.039029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:21.073095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f4a6a2f9fe4b07b9cf74d700d36c3a', 'request_id': 'req-f83d5a1b-3646-4614-aa40-9ec9862d1c22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:21.079520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f4a6a2f9fe4b07b9cf74d700d36c3a', 'request_id': 'req-f83d5a1b-3646-4614-aa40-9ec9862d1c22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:21.090687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f4a6a2f9fe4b07b9cf74d700d36c3a', 'request_id': 'req-f83d5a1b-3646-4614-aa40-9ec9862d1c22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:21.107599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] The token cache is being invalidate because project 54f4a6a2f9fe4b07b9cf74d700d36c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:27:21.107913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:21.108989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f83d5a1b-3646-4614-aa40-9ec9862d1c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:27:21.110835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1179/2368] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:27:20 2022] DELETE /identity/v3/projects/54f4a6a2f9fe4b07b9cf74d700d36c3a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:27:21.115993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.116292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.116543 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.146716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:21.147699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:21.188772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:21.233379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1190/2369] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:21.241633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.242045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.242379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.267606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:21.269462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:21.277815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1180/2370] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:21.282227 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.292695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76f465-b42c-4570-a55a-47005929ca62 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.297301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.297915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.298190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.298459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.299104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:27:21.299403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.301044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.317301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1191/2371] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:27:21.318485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.328205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddabcd9-a304-4851-99e6-b3e0190992aa None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.333578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.334354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.334593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.334822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.335470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.337480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.367137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1181/2372] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:27:21 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:27:21.371391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.382733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b998e-6551-4eb5-b4ad-1377bd5b56ad admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.444500 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.445280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.445380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.445615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.446249 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.448241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.475683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1192/2373] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:27:21 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:27:21.477438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.490774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1a54a-9a57-4915-af94-19e90d383e18 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.496809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.497812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.498094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.498358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.499003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.500609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.513568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1182/2374] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:27:21.518358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.532452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ef8b3-5c1e-49a7-9035-8e10f256de54 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.539079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.539748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.540019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.540296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.541021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.542695 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8da2-d2d2-49cc-a3b2-6f7dc0058807 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.553966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1193/2375] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:27:21.558918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.571298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025765a-5494-42e4-bba5-4e3b5a49ffe5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.590649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.591364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.591644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.591920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] PATH_INFO: `/v3/projects/6e6c1cccd2f043ef8a014f7b9f49b94a/users/481670a490bd404cb924e67aecc80676/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.613397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6c1cccd2f043ef8a014f7b9f49b94a, user_id=481670a490bd404cb924e67aecc80676, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.616217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.641129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1183/2376] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:27:21 2022] PUT /identity/v3/projects/6e6c1cccd2f043ef8a014f7b9f49b94a/users/481670a490bd404cb924e67aecc80676/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:27:21.643078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f335-16cb-42f0-b54d-fe3921a14834 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.643572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f335-16cb-42f0-b54d-fe3921a14834 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.644632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f335-16cb-42f0-b54d-fe3921a14834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.677762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-76d3f335-16cb-42f0-b54d-fe3921a14834 None None] MFA Rules not processed for user `481670a490bd404cb924e67aecc80676`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:21.678987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-76d3f335-16cb-42f0-b54d-fe3921a14834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:21.753317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1194/2377] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:27:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:21.765347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:21.767607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:21.832115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555ec8-bc91-481d-86b2-3b6453f63029 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:21.837539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:21.838113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:21.838391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:21.838667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:21.851527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:21.853388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:21.864860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1184/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:27:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:27:24.744780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade623d0-5e09-4402-ab86-26b9c077282d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:24.745321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade623d0-5e09-4402-ab86-26b9c077282d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:24.745666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade623d0-5e09-4402-ab86-26b9c077282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:24.766418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ade623d0-5e09-4402-ab86-26b9c077282d None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:24.767618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ade623d0-5e09-4402-ab86-26b9c077282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:24.801800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:24.837740 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1195/2379] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:27:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:27:24.869246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:24.870892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:24.934636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:24.959850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb156-647a-4857-a588-49dd89adf250 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:24.966743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:24.966995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:24.967369 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:24.967739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:24.980330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:24.981971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:24.992022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1185/2380] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:27:34.826777 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=9 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:34.988271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:27:55.177738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:55.178309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:55.178309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:55.198803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:27:55.199999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:27:55.207538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1196/2381] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:27:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:27:55.220817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:27:55.231815 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab49f-5293-4651-930e-f696648ffbab service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:27:55.236765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:27:55.237393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:27:55.237669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:27:55.237958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:27:55.246923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:27:55.248525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e23d17-357c-4f8d-9896-3e0192da5b10 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:27:55.256873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1186/2382] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:27:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:28:05.193588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:05.253383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:07.319226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:07.380606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac644f68-8ab3-420f-ac25-1ae7c35ae748 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:07.385660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:07.386361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:07.386633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:07.386944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] PATH_INFO: `/v3/users/481670a490bd404cb924e67aecc80676` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:07.388180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481670a490bd404cb924e67aecc80676)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:07.390257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:07.435294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481670a490bd404cb924e67aecc80676'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.435935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481670a490bd404cb924e67aecc80676'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.442765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481670a490bd404cb924e67aecc80676'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.443393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481670a490bd404cb924e67aecc80676'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.455655 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481670a490bd404cb924e67aecc80676'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.470206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1197/2383] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:28:07 2022] DELETE /identity/v3/users/481670a490bd404cb924e67aecc80676 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:28:07.474645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:07.475148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:07.475258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:07.494009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:28:07.495802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:07.552345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1187/2384] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:28:07.562918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:07.564310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:07.620351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc589d0-70ca-4d8f-a165-ef3681b27981 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:07.623910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:07.624494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:07.624770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:07.625075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:07.633913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:07.635871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650ba4b-1288-4167-a860-c5f0d2f1a031 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:07.645013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1198/2385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:28:07.912413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:07.913852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:07.926908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957123-6b03-4dd8-b437-63d375083880 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:07.932896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:07.933751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:07.934028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:07.934297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] PATH_INFO: `/v3/projects/6e6c1cccd2f043ef8a014f7b9f49b94a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:07.935501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6c1cccd2f043ef8a014f7b9f49b94a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:07.937435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:07.985747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6c1cccd2f043ef8a014f7b9f49b94a', 'request_id': 'req-bc0109c8-6411-4fd9-b129-7bde8654ae79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.991464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6c1cccd2f043ef8a014f7b9f49b94a', 'request_id': 'req-bc0109c8-6411-4fd9-b129-7bde8654ae79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:07.999634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6c1cccd2f043ef8a014f7b9f49b94a', 'request_id': 'req-bc0109c8-6411-4fd9-b129-7bde8654ae79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:08.010988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] The token cache is being invalidate because project 6e6c1cccd2f043ef8a014f7b9f49b94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:28:08.011274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:08.011908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-bc0109c8-6411-4fd9-b129-7bde8654ae79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:08.013432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1188/2386] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:28:07 2022] DELETE /identity/v3/projects/6e6c1cccd2f043ef8a014f7b9f49b94a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:28:08.018284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.018573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.018845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.038325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:28:08.039519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:08.057589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:08.101945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1199/2387] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:28:08.110563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.110890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.111161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.130955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:28:08.132136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:08.140865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1189/2388] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:28:08.145020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.155322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1c06-abe4-4344-a61f-119d316eb207 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.159782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.160371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.160648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.160924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.161594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:28:08.161898 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.163495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.175950 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1200/2389] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:28:08.180066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.189618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9082782-7702-44b6-a3ee-51218fa01aea None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.194113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.194694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.194966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.195237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.195857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.198035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.225680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1190/2390] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:28:08.230661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.240428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51009c-9c31-44f3-966d-9571e93a12ef admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.300979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.301694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.302020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.302373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.303046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.305133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.327144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1201/2391] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:28:08 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:28:08.331315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.342136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3c3c-a923-40f7-aad4-48284094945b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.346695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.347277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.347554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.347822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.348448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.350082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.360453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1191/2392] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:28:08.364185 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.374047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875eefb-cad2-4993-9e88-d278dca1c7e2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.378694 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.379283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.379560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.379863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.380499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.382141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab57cf-5612-4735-a2f2-2c3aa16a4b20 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.392703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1202/2393] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:28:08.396776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.407273 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088a74-21dd-441b-95ee-8932530be48f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.411943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.412590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.412868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.413170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] PATH_INFO: `/v3/projects/f42a09e1aed94e74bd40ffcaded98dad/users/7f5861d7d27949e295a0af04e6bf2cf5/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.428162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42a09e1aed94e74bd40ffcaded98dad, user_id=7f5861d7d27949e295a0af04e6bf2cf5, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.430829 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.444493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1192/2394] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:28:08 2022] PUT /identity/v3/projects/f42a09e1aed94e74bd40ffcaded98dad/users/7f5861d7d27949e295a0af04e6bf2cf5/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:28:08.456070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.456360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.456636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.485501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] MFA Rules not processed for user `7f5861d7d27949e295a0af04e6bf2cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:28:08.486700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:08.550598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1203/2395] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:28:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:28:08.561766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:08.563497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:08.585213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:08.622158 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c730b-a051-4bcd-95eb-56e165f79d2b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:08.625765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:08.626361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:08.626660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:08.626931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:08.635495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:08.637036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62164-f088-4051-9007-a94e20fa479b service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:08.645703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1193/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:28:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:28:18.547706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:18.641667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=17 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:18.829467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:18.890599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcb95c-707e-44ac-83a2-9ba62fb4b1e3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:18.895080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:18.895682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:18.895957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:18.896248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] PATH_INFO: `/v3/users/7f5861d7d27949e295a0af04e6bf2cf5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:18.897372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5861d7d27949e295a0af04e6bf2cf5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:18.899331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:18.944050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5861d7d27949e295a0af04e6bf2cf5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:18.944690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5861d7d27949e295a0af04e6bf2cf5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:18.951585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5861d7d27949e295a0af04e6bf2cf5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:18.952206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5861d7d27949e295a0af04e6bf2cf5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:18.964467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5861d7d27949e295a0af04e6bf2cf5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:18.976555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1204/2397] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:28:18 2022] DELETE /identity/v3/users/7f5861d7d27949e295a0af04e6bf2cf5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:28:18.981598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:18.981883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:18.982155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:19.000855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:28:19.001996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:19.059884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1194/2398] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:28:19.069494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:19.070885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:19.127546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f83ea-7c9b-4b79-b824-34caab1832f1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:19.131192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:19.131788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:19.132065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:19.132333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:19.141747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:19.143164 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:19.153216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1205/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:28:19.415983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:19.417747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:19.430478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cb73a-8e48-4ba0-b447-9e97280a3e16 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:19.441190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:19.441835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:19.442106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:19.442523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] PATH_INFO: `/v3/projects/f42a09e1aed94e74bd40ffcaded98dad` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:19.443703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42a09e1aed94e74bd40ffcaded98dad)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:19.445645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:19.491712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42a09e1aed94e74bd40ffcaded98dad', 'request_id': 'req-b589dc23-c1b0-4922-a31b-3d35f2947dcd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:19.497741 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42a09e1aed94e74bd40ffcaded98dad', 'request_id': 'req-b589dc23-c1b0-4922-a31b-3d35f2947dcd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:19.505198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42a09e1aed94e74bd40ffcaded98dad', 'request_id': 'req-b589dc23-c1b0-4922-a31b-3d35f2947dcd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:19.517125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] The token cache is being invalidate because project f42a09e1aed94e74bd40ffcaded98dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:28:19.517414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:19.518045 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:28:19.519511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1195/2400] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:28:19 2022] DELETE /identity/v3/projects/f42a09e1aed94e74bd40ffcaded98dad => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:28:29.148334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:29.514174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:28:56.612270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:56.622643 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af344be7-507a-4ea5-af55-ac4f602a5691 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:56.678489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:56.679125 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:56.679397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:56.679669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:56.681667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:56.769254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:28:56.770728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:56.772826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:56.774311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899afa9b-4a59-406e-b5a8-07a32b74a295 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:56.783953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589dc23-c1b0-4922-a31b-3d35f2947dcd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:28:56.786341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1206/2401] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:28:56.787753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:28:56.788356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:28:56.788630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:28:56.788897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:28:56.790613 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:28:56.804660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:28:56.806207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3c1b-680a-479c-89cf-4c830cdacf3e service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:28:56.817392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1196/2402] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:29:04.103020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:04.103545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:04.103545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:04.124654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] MFA Rules not processed for user `d286b1e0afe9413291e0e2503b4c204a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:04.126666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:04.190492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1207/2403] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:05.211054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:05.211577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:05.211577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:05.232531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] MFA Rules not processed for user `764cfe23613e4a97ba05f8bf0ed77ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:05.234035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:05.290295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1197/2404] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:14.182412 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=79 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:15.283667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=31 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:20.321456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:20.322966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:20.338834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b127baf-c76e-410e-a2f5-0d65fa4c2915 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:20.344993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:20.345592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:20.345868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:20.346139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:20.347779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:20.358912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:20.360485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:20.369321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1208/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:30.365259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:33.966224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:33.982887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d543ff-73d4-4df1-a3cd-0ae9d13d3a97 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:33.990775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:33.991414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:33.991757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:33.992267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:33.994221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:34.009206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:34.010913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62447b9f-13ca-4aa1-931c-e89b7af48a2b service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:34.023331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1198/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:34.108599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:34.126220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cd3b7-cdc0-4d21-93ba-d5c87a489e79 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:34.132308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:34.133488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:34.134871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:34.135393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:34.137662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:34.148834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:34.150341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:34.159488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1209/2407] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:42.324729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b7fc-8624-4954-bddb-a3f8c1f93b4f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:42.325196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b7fc-8624-4954-bddb-a3f8c1f93b4f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:42.325318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b7fc-8624-4954-bddb-a3f8c1f93b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:42.345551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b5d6b7fc-8624-4954-bddb-a3f8c1f93b4f None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:42.346743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b5d6b7fc-8624-4954-bddb-a3f8c1f93b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:42.400654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1199/2408] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:42.448560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:42.449975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:42.519399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e4663-a665-4d50-9246-048e8a8b2f4f service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:42.524264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:42.524858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:42.525160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:42.525426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:42.533581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:42.536128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9efc90-e9e6-4ca8-ad8c-381798d29b82 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:42.544001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1210/2409] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:29:45.914453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493912-fd2b-4d3c-9652-5a06e216ba75 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:45.914955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493912-fd2b-4d3c-9652-5a06e216ba75 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:45.915070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493912-fd2b-4d3c-9652-5a06e216ba75 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:45.916481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1200/2410] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 07:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:29:45.922328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:45.922850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:45.922850 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:45.945415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:45.946776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:46.027331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1211/2411] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 07:29:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:46.032583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493912-fd2b-4d3c-9652-5a06e216ba75 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:46.046259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493912-fd2b-4d3c-9652-5a06e216ba75 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:46.051333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:46.052012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:46.052275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:46.052540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:46.054297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:46.066621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:46.068411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:46.079017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1201/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:49.269447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:49.271334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:49.341092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6579d-2501-4045-8993-449f0c45290d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:49.346075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:49.346700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:49.346978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:49.347248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] PATH_INFO: `/v3/users/06451ea3cd91471c9b09966516090509` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:49.348351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=06451ea3cd91471c9b09966516090509)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:49.350324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:49.363759 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:49.397293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06451ea3cd91471c9b09966516090509'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.397961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06451ea3cd91471c9b09966516090509'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.405053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06451ea3cd91471c9b09966516090509'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.406113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06451ea3cd91471c9b09966516090509'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.418932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06451ea3cd91471c9b09966516090509'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.431198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1212/2413] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:29:49 2022] DELETE /identity/v3/users/06451ea3cd91471c9b09966516090509 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:49.602536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:49.617520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e8c39-60c4-4729-a5f1-7b3c2876906f service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:49.675625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:49.676255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:49.676534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:49.676806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] PATH_INFO: `/v3/projects/a3324b12312f43ebab7273b438ad9b65` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:49.677949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3324b12312f43ebab7273b438ad9b65)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:49.679838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:49.730657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3324b12312f43ebab7273b438ad9b65', 'request_id': 'req-808829a0-f7ff-4a45-9f2e-18500795a793'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.736569 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3324b12312f43ebab7273b438ad9b65', 'request_id': 'req-808829a0-f7ff-4a45-9f2e-18500795a793'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.743747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3324b12312f43ebab7273b438ad9b65', 'request_id': 'req-808829a0-f7ff-4a45-9f2e-18500795a793'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.755646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] The token cache is being invalidate because project a3324b12312f43ebab7273b438ad9b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:29:49.755934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.756612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.758109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1202/2414] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:29:49 2022] DELETE /identity/v3/projects/a3324b12312f43ebab7273b438ad9b65 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:49.762166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:49.763577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:49.776337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4355a-7471-41f1-9182-86daceae0eba admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:49.832896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:49.833476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:49.833751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:49.834024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] PATH_INFO: `/v3/users/f552039a4b76474991367034bb909723` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:49.835161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f552039a4b76474991367034bb909723)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:49.837130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:49.883149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f552039a4b76474991367034bb909723'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.883793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f552039a4b76474991367034bb909723'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.891385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f552039a4b76474991367034bb909723'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.892016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f552039a4b76474991367034bb909723'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.904141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-20972805-4215-4a3e-a621-42703254f873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f552039a4b76474991367034bb909723'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:49.916925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1213/2415] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:29:49 2022] DELETE /identity/v3/users/f552039a4b76474991367034bb909723 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:50.092129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.094330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:50.106909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808829a0-f7ff-4a45-9f2e-18500795a793 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.112379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:50.165229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.165854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.166131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.166401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] PATH_INFO: `/v3/projects/45f7438a4c6f4d28a9776154a47aa1de` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.167499 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f7438a4c6f4d28a9776154a47aa1de)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.169437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.218807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f7438a4c6f4d28a9776154a47aa1de', 'request_id': 'req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:50.224342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f7438a4c6f4d28a9776154a47aa1de', 'request_id': 'req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:50.231125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f7438a4c6f4d28a9776154a47aa1de', 'request_id': 'req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:50.243343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] The token cache is being invalidate because project 45f7438a4c6f4d28a9776154a47aa1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:29:50.243625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:50.244267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9199fa0f-41c3-47c8-8d3c-068cbfc2ac54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:29:50.245759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1203/2416] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:29:50 2022] DELETE /identity/v3/projects/45f7438a4c6f4d28a9776154a47aa1de => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:50.250639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.250945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.251248 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.270358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:50.271534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:50.276960 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:50.337007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1214/2417] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:50.345482 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.345762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.346029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.366135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:50.367351 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:50.375581 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1204/2418] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:50.379611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.389908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176a258-a495-46c4-aa1a-909c4cd2b254 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.394810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.395408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.395682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.395946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.396582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:29:50.396880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.398496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.411081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1215/2419] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:29:50.415478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.424666 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90926b37-cc32-4819-9cc9-79825e23d106 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.429294 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.429881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.430157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.430424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.431058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.432933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.460347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1205/2420] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:29:50 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:29:50.464502 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.475119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ac70-23e0-46dc-a56f-5a3b9a7a010b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.531187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.531788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.532066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.532335 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.533006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.534982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.555679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1216/2421] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:29:50 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:29:50.560071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.570270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b94dfe-1a07-4849-b189-9289abaf4ac3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.574848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.575434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.575710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.575976 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.576607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.578722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.588926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1206/2422] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:29:50.593494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.605158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169319-32b4-4649-be9a-0567c4957abd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.610122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.610714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.610992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.611261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.611892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.613521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75683a0a-4188-490b-85b1-3d9d2a4ae958 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.623944 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1217/2423] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:29:50.628267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.637978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e985-c42f-4721-aa83-d8165baccc87 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.643500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.644136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.644413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.644681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] PATH_INFO: `/v3/projects/bfa33399f5994fb4a1ad804fa6e75f8b/users/04cf365443b44d9d87338d95b5b0251b/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.660079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa33399f5994fb4a1ad804fa6e75f8b, user_id=04cf365443b44d9d87338d95b5b0251b, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.662809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.675668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1207/2424] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:29:50 2022] PUT /identity/v3/projects/bfa33399f5994fb4a1ad804fa6e75f8b/users/04cf365443b44d9d87338d95b5b0251b/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:50.681008 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb68f70-f785-413d-8390-b4ec4e0d12dc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.681297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb68f70-f785-413d-8390-b4ec4e0d12dc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.681573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb68f70-f785-413d-8390-b4ec4e0d12dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.702540 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fdb68f70-f785-413d-8390-b4ec4e0d12dc None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:50.703803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fdb68f70-f785-413d-8390-b4ec4e0d12dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:50.762321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1218/2425] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:50.773530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:50.775479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:50.822486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:50.844101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a313528-dd98-45bc-b974-2c674dec80ec admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:50.847781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:50.848755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:50.848755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:50.848931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:50.858715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:50.860133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:50.869764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1208/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:54.465790 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.466312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.466312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.500747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] MFA Rules not processed for user `04cf365443b44d9d87338d95b5b0251b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:54.502017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:54.507311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:54.578878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1219/2427] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:54.583563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.594178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26acab3-7952-4bb7-93e6-8793e20e2e49 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.599092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.599696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.600150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.600243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.600866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.602801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.635961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1209/2428] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:29:54 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:29:54.640297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.651326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694f8ff-625b-4d68-a963-baad5ea6f18f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.718720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.719422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.719729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.719997 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.720642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.722591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.747818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1220/2429] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:29:54 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:29:54.749702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.766027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95d991-c5eb-437c-a532-c63775bdc117 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.770955 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.771573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.771844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.772116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.772757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.774470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.787634 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1210/2430] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:29:54.792103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.802936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6ae42-5651-48e0-a5b6-eb66bdb88746 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.808365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.809490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.809910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.810191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] PATH_INFO: `/v3/projects/861b07d115dd445786956f32e8c94261/users/0a672af929f3442e88d31d2d8dc06ed9/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.835942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=861b07d115dd445786956f32e8c94261, user_id=0a672af929f3442e88d31d2d8dc06ed9, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.839287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.854796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1221/2431] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:29:54 2022] PUT /identity/v3/projects/861b07d115dd445786956f32e8c94261/users/0a672af929f3442e88d31d2d8dc06ed9/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:29:54.859819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.870449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6085d-7b7f-41d5-bc71-4d0de6cf0a6b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.941681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.942313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.942591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.942857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.943528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.945201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.956025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1211/2432] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:29:54.960126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:54.971369 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc66c4-5003-454b-ba64-8149c135b97d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:54.977278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:54.978088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:54.978365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:54.978637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] PATH_INFO: `/v3/projects/861b07d115dd445786956f32e8c94261/users/0a672af929f3442e88d31d2d8dc06ed9/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:54.980579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=861b07d115dd445786956f32e8c94261, user_id=0a672af929f3442e88d31d2d8dc06ed9, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:54.984087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa02a3-87f4-4389-bdfb-3710d1d17baa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:54.995457 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1222/2433] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:29:54 2022] PUT /identity/v3/projects/861b07d115dd445786956f32e8c94261/users/0a672af929f3442e88d31d2d8dc06ed9/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:29:57.717651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:57.721036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:57.799576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93989926-8c87-4fa4-a7c3-bfcd58f54ef4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:57.869991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c4779-7631-481b-958f-54615871be22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:57.870609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4779-7631-481b-958f-54615871be22 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:57.871320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4779-7631-481b-958f-54615871be22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:57.871320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4779-7631-481b-958f-54615871be22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:57.874237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-587c4779-7631-481b-958f-54615871be22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:57.887494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:58.000642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c4779-7631-481b-958f-54615871be22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:58.002900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c4779-7631-481b-958f-54615871be22 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:58.016435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1212/2434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:29:59.571334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:59.572127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:59.572559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:59.624724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] MFA Rules not processed for user `0a672af929f3442e88d31d2d8dc06ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:29:59.626241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:59.695493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:29:59.743024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1223/2435] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:29:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:29:59.756333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c4779-7631-481b-958f-54615871be22 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:29:59.757903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-587c4779-7631-481b-958f-54615871be22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:29:59.870248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c4779-7631-481b-958f-54615871be22 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:29:59.876892 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:29:59.877534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:29:59.877808 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:29:59.878075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:29:59.951521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:29:59.954427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:29:59.966375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1213/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:29:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:05.674515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:05.689551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9243b-99c6-4c48-9525-c719cfd733fb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:05.695930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:05.697711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:05.697711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:05.697711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:05.698925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:05.724054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:05.726168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8168-4f05-4e54-8c0e-1cfda7fa8657 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:05.743128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1224/2437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:09.335498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:09.337059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:09.349427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea764a-983f-428b-a347-765ccfb8885d service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:09.353224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:09.353853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:09.354127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:09.354392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:09.356186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:09.408643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:09.424275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:09.425766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:09.435171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1214/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:15.738351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:17.525115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:17.525708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:17.525708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:17.545896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] MFA Rules not processed for user `269ef35368734dcd9d3cc901db39fe10`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:17.547085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:17.553957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1225/2439] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:17.867017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:17.868424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:17.930145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a5aa-95e9-494c-b7dc-963abe911cb0 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:17.933768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:17.934623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:17.934897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:17.935166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:17.937187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:17.946519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:17.947918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6412-fd18-4968-9bd3-f512024682e9 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:17.955783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1215/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:18.266341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:18.267799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:18.340856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b46b26-e306-4191-86d3-8a02a2ff0587 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:18.346024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:18.346625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:18.346901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:18.347171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:18.348855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:18.360628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:18.362230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:18.371774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1226/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:20.768921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:20.769516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:20.769516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:20.789898 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] MFA Rules not processed for user `04cf365443b44d9d87338d95b5b0251b`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:20.791093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:20.797821 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1216/2442] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:21.351504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:21.360551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4141c5-a0ba-4df9-8f06-9863f25c5304 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:21.364135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:21.364725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:21.365029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:21.365298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:21.367012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:21.446466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:21.447912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:21.458302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1227/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:21.521996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:21.533623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe82018-e395-4ee9-b972-e7720a7abdc3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:21.538938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:21.541493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:21.541782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:21.543393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:21.545289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:21.562884 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:21.565411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:21.575715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1217/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:25.606461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:25.608451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:25.620413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb13e9-2f3d-4072-bc0b-666558359f8e service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:25.631975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:25.632627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:25.632906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:25.633212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] PATH_INFO: `/v3/users/d286b1e0afe9413291e0e2503b4c204a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:25.634360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d286b1e0afe9413291e0e2503b4c204a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:25.636298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:25.684304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd286b1e0afe9413291e0e2503b4c204a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:25.685221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd286b1e0afe9413291e0e2503b4c204a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:25.701592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd286b1e0afe9413291e0e2503b4c204a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:25.702373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd286b1e0afe9413291e0e2503b4c204a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:25.714184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:25.715603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b97251cb-0e26-4882-b1df-a1df47559797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd286b1e0afe9413291e0e2503b4c204a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:25.735539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1228/2445] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:30:25 2022] DELETE /identity/v3/users/d286b1e0afe9413291e0e2503b4c204a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:30:25.961270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:25.976606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6200b5f-41d3-4cca-87cc-4d35e02e2048 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.107153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.109385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.109944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.110554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] PATH_INFO: `/v3/projects/fc7675d275394d8893d0facf20d3a4e7` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.112080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7675d275394d8893d0facf20d3a4e7)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.115191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.191434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7675d275394d8893d0facf20d3a4e7', 'request_id': 'req-280e7af6-5ae5-4937-ae62-95dceb34cff1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:26.198102 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7675d275394d8893d0facf20d3a4e7', 'request_id': 'req-280e7af6-5ae5-4937-ae62-95dceb34cff1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:26.205991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7675d275394d8893d0facf20d3a4e7', 'request_id': 'req-280e7af6-5ae5-4937-ae62-95dceb34cff1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:26.215182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:26.219857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] The token cache is being invalidate because project fc7675d275394d8893d0facf20d3a4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:30:26.220352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:26.221098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-280e7af6-5ae5-4937-ae62-95dceb34cff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:30:26.223233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1218/2446] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:30:25 2022] DELETE /identity/v3/projects/fc7675d275394d8893d0facf20d3a4e7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:30:26.228440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.228727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.229024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.248856 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:26.250091 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:26.323420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1229/2447] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:26.327575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.328429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.328711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.353516 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:26.354748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:26.373794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.374070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1219/2448] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:26.385732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236f0ce-f6c4-4601-8f87-9e464b624bd4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.407888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.411859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.412198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.412472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.413188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:30:26.413637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.415135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.440126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1230/2449] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:30:26.445124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.455655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bfcb8-91eb-43ef-a1e6-b0546ad0c9d7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.461124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.461773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.462290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.462803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.463548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.465704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.501015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1220/2450] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:30:26 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:30:26.503455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.513392 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64589a-5d41-4c86-befb-047faba0aed8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.571862 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.572468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.572751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.573086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.573718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.575693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.604041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1231/2451] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:30:26 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:30:26.608334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.619681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920be3a-2e01-4727-81aa-bac9f3c2cf29 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.625982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.626591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.627037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.627463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.628457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.630902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.642944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1221/2452] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:30:26.646985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.660259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f61227-4169-49b0-b2d8-eb07a03be3d0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.665042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.665611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.665887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.666154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.666788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.668373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f974e-64f6-4d5a-8406-b54d2c287327 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.679640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1232/2453] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:30:26.683931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.698494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5649-5074-4f18-9ab7-de20c317efb7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.706609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.707252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.707527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.707797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] PATH_INFO: `/v3/projects/94a0b09b62ce47678fd542a0453c3d17/users/44adfe9a391948f68dc747448546063e/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.724104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a0b09b62ce47678fd542a0453c3d17, user_id=44adfe9a391948f68dc747448546063e, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.726907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.743206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1222/2454] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:30:26 2022] PUT /identity/v3/projects/94a0b09b62ce47678fd542a0453c3d17/users/44adfe9a391948f68dc747448546063e/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:30:26.749031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.749529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.750539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.771584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:26.772792 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:26.827544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:26.843803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1233/2455] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:30:26.853215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:26.855113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:26.911269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf00382-160d-4102-ba8d-58cd7b9888e2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:26.914846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:26.915471 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:26.915744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:26.916016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:26.925122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:26.926533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63956b6a-9bfb-4e95-af99-6d2383076444 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:26.936404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1223/2456] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:28.005733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:28.009451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:28.127463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e357f-567d-473e-8a3d-aec5c1737597 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:28.133386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:28.134105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:28.134475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:28.134811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:28.136672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:28.150287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:28.152043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:28.171155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1234/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:32.125189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5e11-9fca-41cd-ab73-ef9160e84e99 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:32.125688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5e11-9fca-41cd-ab73-ef9160e84e99 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:32.125800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5e11-9fca-41cd-ab73-ef9160e84e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:32.168780 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2c0f5e11-9fca-41cd-ab73-ef9160e84e99 None None] MFA Rules not processed for user `44adfe9a391948f68dc747448546063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:32.172712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2c0f5e11-9fca-41cd-ab73-ef9160e84e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:32.303323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1224/2458] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:32.313634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:32.315187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:32.335871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325795-1ace-45e8-8e49-7136d814a79a service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:32.345481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:32.346246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:32.346628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:32.347000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:32.357137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:32.359563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:32.381903 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1235/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:35.071722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1ee30-8ac7-427e-bb14-bfac8092eaa8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:35.072169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1ee30-8ac7-427e-bb14-bfac8092eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:35.072517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1ee30-8ac7-427e-bb14-bfac8092eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:35.102924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dcc1ee30-8ac7-427e-bb14-bfac8092eaa8 None None] MFA Rules not processed for user `44adfe9a391948f68dc747448546063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:35.104296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dcc1ee30-8ac7-427e-bb14-bfac8092eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:35.113712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1225/2460] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:30:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:39.978287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:39.979743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:40.006555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37e42-caa8-4d8c-8e85-a1c78722a675 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:40.010710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:40.011484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:40.011988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:40.012270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:40.014571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:40.089278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:40.102899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:40.129378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1236/2461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:30:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:30:44.222454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03ad8-2074-443e-9745-687a3c3bc4fe None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:44.222961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03ad8-2074-443e-9745-687a3c3bc4fe None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:44.223290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03ad8-2074-443e-9745-687a3c3bc4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:44.245683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-79b03ad8-2074-443e-9745-687a3c3bc4fe None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:30:44.246953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-79b03ad8-2074-443e-9745-687a3c3bc4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:44.265836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:44.320078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1226/2462] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:30:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:30:44.374219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:30:44.375714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:30:44.445370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:44.450407 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d941e2-bb7f-45f6-9617-2c86819cbf24 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:30:44.455870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:30:44.456476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:30:44.456753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:30:44.457054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:30:44.465467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:30:44.467106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744fc84-1e03-4a12-93dd-dee473cc2bf8 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:30:44.480136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1237/2463] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:30:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:30:54.310956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=15 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:30:54.471868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:31:00.005739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.006303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.006401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.007894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1227/2464] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:31:00.014673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93d9de-b044-4366-9808-bd56842a1fb7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.015169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93d9de-b044-4366-9808-bd56842a1fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.015280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93d9de-b044-4366-9808-bd56842a1fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.035283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3a93d9de-b044-4366-9808-bd56842a1fb7 None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:00.036491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3a93d9de-b044-4366-9808-bd56842a1fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:00.046442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1238/2465] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:31:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:31:00.058782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:31:00.061086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:00.118870 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e48b50-e808-4bf7-ab8a-b11c93b85c58 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:31:00.122494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:31:00.123227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.123504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.123769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.134060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:31:00.135473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5065b6-b2f4-47b4-926a-4d473d872c96 service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:31:00.152240 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1228/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:31:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:31:00.226205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1fd3-313c-43d0-9cf7-861fb6738c3e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.226491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1fd3-313c-43d0-9cf7-861fb6738c3e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.226791 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1fd3-313c-43d0-9cf7-861fb6738c3e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.228160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1239/2467] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 07:31:00.232978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363a6f8-0563-4e7a-a9c0-837b7aea313c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.233267 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363a6f8-0563-4e7a-a9c0-837b7aea313c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.233540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363a6f8-0563-4e7a-a9c0-837b7aea313c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.252968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d363a6f8-0563-4e7a-a9c0-837b7aea313c None None] MFA Rules not processed for user `170658d3fbf547a09a4dd2bb77ecb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:00.254147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d363a6f8-0563-4e7a-a9c0-837b7aea313c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:00.260511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1229/2468] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:31:00.272707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1fd3-313c-43d0-9cf7-861fb6738c3e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:31:00.284088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1fd3-313c-43d0-9cf7-861fb6738c3e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:31:00.287621 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:31:00.288148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.288420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.288689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.295687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:31:00.297140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131d0ce-e7d7-4ac6-8467-cbd8351d0ff4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:31:00.304661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1240/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:31:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:31:00.385893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d2f2-cfb0-4b73-bbe4-272607386aca None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.386171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d2f2-cfb0-4b73-bbe4-272607386aca None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.386564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d2f2-cfb0-4b73-bbe4-272607386aca None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.387995 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1230/2470] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:31:00.393972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5ade-7834-4187-a637-158093e2447f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.394259 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5ade-7834-4187-a637-158093e2447f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.394534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5ade-7834-4187-a637-158093e2447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.414607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e5cf5ade-7834-4187-a637-158093e2447f None None] MFA Rules not processed for user `8e403816a5674d45b3cefdc34cffcf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:00.415873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e5cf5ade-7834-4187-a637-158093e2447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:00.425470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1241/2471] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:31:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:31:00.436426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381d2f2-cfb0-4b73-bbe4-272607386aca None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:31:00.447283 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381d2f2-cfb0-4b73-bbe4-272607386aca None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:31:00.450883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:31:00.452159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:00.452436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:00.452711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:00.462873 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:31:00.464640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:31:00.475501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1231/2472] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:31:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:31:02.788813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f5e0d-c6d9-4a3e-9733-5dafa02b3cdd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:02.789403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f5e0d-c6d9-4a3e-9733-5dafa02b3cdd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:02.789403 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f5e0d-c6d9-4a3e-9733-5dafa02b3cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:02.808280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8a5f5e0d-c6d9-4a3e-9733-5dafa02b3cdd None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:02.809487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8a5f5e0d-c6d9-4a3e-9733-5dafa02b3cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:02.817569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1242/2473] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:31:02.830099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:31:02.841311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3043b-2cde-449d-9c81-558c798cd4dd service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:31:02.846466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:31:02.846987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:02.847258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:02.847523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:02.855625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:31:02.857219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:31:02.865136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1232/2474] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:31:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:31:12.803583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=15 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:31:12.861986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=41 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:31:22.950265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1d42-801c-4dca-9343-d266067f043b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:22.950806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1d42-801c-4dca-9343-d266067f043b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:22.950806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1d42-801c-4dca-9343-d266067f043b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:22.971662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d6ed1d42-801c-4dca-9343-d266067f043b None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:22.972827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d6ed1d42-801c-4dca-9343-d266067f043b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:22.979981 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1243/2475] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:31:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:31:22.992400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:31:23.002881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da24a-2f0d-4139-b6b5-af8a7583e5a9 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:31:23.007757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:31:23.008336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:23.008608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:23.008875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:23.017327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:31:23.018864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:31:23.026765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1233/2476] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:31:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:31:32.965805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:31:33.023564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:31:57.375387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43733d-6145-4f11-bb07-e5366ccc8737 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:31:57.375905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43733d-6145-4f11-bb07-e5366ccc8737 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:31:57.376004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43733d-6145-4f11-bb07-e5366ccc8737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:31:57.396367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ed43733d-6145-4f11-bb07-e5366ccc8737 None None] MFA Rules not processed for user `44adfe9a391948f68dc747448546063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:31:57.397597 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed43733d-6145-4f11-bb07-e5366ccc8737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:31:57.404406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1244/2477] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:31:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:06.016187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:06.035961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbe0bc-098b-4d63-bb31-72ab539732f0 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:06.039750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:06.040356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:06.040635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:06.040920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:06.042653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:06.106949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:06.108405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704b1c4-e33b-4262-abd8-b7dbeea69bd8 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:06.127888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1234/2478] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:07.391073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:09.663533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:09.664581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:09.664994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:09.666729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1245/2479] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 07:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 07:32:09.675742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:09.676638 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:09.677022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:09.763215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:09.765448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:09.774754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] MFA Rules not processed for user `b1cbc4b10d4745d08bb05dbd7a36dc84`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:09.776217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:09.793154 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc4723-92fb-49e2-92ea-a393a0f7358d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:09.798983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:09.800733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:09.801049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:09.801329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] PATH_INFO: `/v3/users/269ef35368734dcd9d3cc901db39fe10` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:09.802697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] RBAC: Authorizing `identity:delete_user(user_id=269ef35368734dcd9d3cc901db39fe10)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:09.804626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:09.879512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1235/2480] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 07:32:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:32:09.894323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:09.898022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:09.914549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd958714-2850-49b6-b45f-3d5a8102cad5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:09.920347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:09.921153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:09.921559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:09.921930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:09.933152 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:09.936042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:09.949337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1236/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:32:09.990940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269ef35368734dcd9d3cc901db39fe10'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:09.991669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269ef35368734dcd9d3cc901db39fe10'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:09.998676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269ef35368734dcd9d3cc901db39fe10'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:09.999317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269ef35368734dcd9d3cc901db39fe10'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.011802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b74a3db4-98cc-444d-824c-4ee8e71f032b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269ef35368734dcd9d3cc901db39fe10'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.025700 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1246/2482] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:09 2022] DELETE /identity/v3/users/269ef35368734dcd9d3cc901db39fe10 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:10.201839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.207396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:10.220537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958aa780-adbd-4f68-9bc4-2df85fe2d853 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.296554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.297326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.297707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.298082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] PATH_INFO: `/v3/projects/6f391a1187074b68b3c6a887634b479b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.299314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f391a1187074b68b3c6a887634b479b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.302111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.353194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f391a1187074b68b3c6a887634b479b', 'request_id': 'req-79945dd9-59f1-4192-9499-e757b2bb8d25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.365566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f391a1187074b68b3c6a887634b479b', 'request_id': 'req-79945dd9-59f1-4192-9499-e757b2bb8d25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.373464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f391a1187074b68b3c6a887634b479b', 'request_id': 'req-79945dd9-59f1-4192-9499-e757b2bb8d25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.388352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] The token cache is being invalidate because project 6f391a1187074b68b3c6a887634b479b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:10.388744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.389342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79945dd9-59f1-4192-9499-e757b2bb8d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:10.391769 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1237/2483] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:10 2022] DELETE /identity/v3/projects/6f391a1187074b68b3c6a887634b479b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:10.396720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.396969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.397244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.452932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:10.454317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:10.522399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1247/2484] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:10.531213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.531807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.532082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.552984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:10.554202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:10.562745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1238/2485] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:32:10.567049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.579301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010706f-27f7-4bc7-aa32-e2c6a2b32c76 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.586365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.587686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.588138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.588576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.589476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:10.590542 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.593424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.608252 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1248/2486] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 07:32:10.613243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.622630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ddd7e-8620-4973-8128-c4d2ffa45ca0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.627612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.628208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.628485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.628754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.629433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.631317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.658695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1239/2487] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:10 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:10.664240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.675133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7651ea-11b6-4781-9449-7ad0b4584a0e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.717863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:10.748690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.749332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.749606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.749878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.750520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.752481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.779993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1249/2488] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:10 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:10.781858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.793397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529d7b-785d-4362-b4b8-000667da6de2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.798180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.798816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.799096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.799371 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.801413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.803029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.814907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1240/2489] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:10.818804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.830634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e38dc2-ceb9-4a42-94d9-df575e8ae3db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.837398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.838076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.838357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.838627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.839271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.841157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d733-5bb7-4523-a2fa-9feac00baa0d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.852281 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1250/2490] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:10.856932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:10.867644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4546-c5dc-472b-9880-3f1ed277a2fc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:10.872350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:10.873028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.873306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.873574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] PATH_INFO: `/v3/projects/d7dbaee9c6f24c12b6113473f42cf134/users/62e2c7b6c2494b28b80bb2da789a0f61/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.897201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7dbaee9c6f24c12b6113473f42cf134, user_id=62e2c7b6c2494b28b80bb2da789a0f61, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:10.899953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:10.929006 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1241/2491] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:10 2022] PUT /identity/v3/projects/d7dbaee9c6f24c12b6113473f42cf134/users/62e2c7b6c2494b28b80bb2da789a0f61/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:10.932262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:10.932574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:10.932842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:10.967123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] MFA Rules not processed for user `62e2c7b6c2494b28b80bb2da789a0f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:10.968337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:11.055854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1251/2492] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:11.067418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:11.069170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:11.109242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:11.193708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a70129-06c2-4d85-8344-6cb7fc14b16e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:11.200656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:11.201385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:11.201664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:11.201932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:11.211661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:11.213718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21cb5d-522e-4275-b731-479bc9efb0cf service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:11.224301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1242/2493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:13.943572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.005998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ad83-ae99-45e5-b00d-b588d27bd35a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.010589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.011219 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.011493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.011762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] PATH_INFO: `/v3/users/62e2c7b6c2494b28b80bb2da789a0f61` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.012879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e2c7b6c2494b28b80bb2da789a0f61)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.014833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.065800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e2c7b6c2494b28b80bb2da789a0f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.066582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e2c7b6c2494b28b80bb2da789a0f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.067854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:14.079223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e2c7b6c2494b28b80bb2da789a0f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.079956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e2c7b6c2494b28b80bb2da789a0f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.098414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e2c7b6c2494b28b80bb2da789a0f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.116732 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1252/2494] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:13 2022] DELETE /identity/v3/users/62e2c7b6c2494b28b80bb2da789a0f61 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:14.121486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.122034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.122034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.141910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:14.143659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.207339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1243/2495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:32:14.213816 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.215415 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.274425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38aab4c-9b23-4e0c-82fa-7a5bdd0606b0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.278385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.278989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.279288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.279557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.289391 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.290867 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.301158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1253/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:14.588901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.590862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.604637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34232f82-4779-4fc6-90a4-d9d821bcdbd9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.609585 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.610245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.610919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.610919 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] PATH_INFO: `/v3/projects/d7dbaee9c6f24c12b6113473f42cf134` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.612684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7dbaee9c6f24c12b6113473f42cf134)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.614876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.645036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.647279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.675452 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7dbaee9c6f24c12b6113473f42cf134', 'request_id': 'req-90ca64f7-4513-4238-8821-44756cb13583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.681650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7dbaee9c6f24c12b6113473f42cf134', 'request_id': 'req-90ca64f7-4513-4238-8821-44756cb13583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.692740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7dbaee9c6f24c12b6113473f42cf134', 'request_id': 'req-90ca64f7-4513-4238-8821-44756cb13583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.717973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] The token cache is being invalidate because project d7dbaee9c6f24c12b6113473f42cf134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:14.718736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.719620 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-90ca64f7-4513-4238-8821-44756cb13583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:14.722289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1244/2497] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:14 2022] DELETE /identity/v3/projects/d7dbaee9c6f24c12b6113473f42cf134 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:14.732979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8584f-6662-4f2e-839b-d502c3617b62 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.733335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8584f-6662-4f2e-839b-d502c3617b62 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.733684 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8584f-6662-4f2e-839b-d502c3617b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.760956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-01d8584f-6662-4f2e-839b-d502c3617b62 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:14.762193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01d8584f-6662-4f2e-839b-d502c3617b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.769855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:14.777685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b65a0-0e9f-4d42-8866-585c5d686f39 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.848694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1245/2498] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:14.851551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.851842 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.852284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.867693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.868449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.869173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.869173 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.871041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.872980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:14.874222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:14.883572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.887139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1246/2499] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:32:14.887423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.890463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.894304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:14.902031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1254/2500] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:14.905938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7895aae-98a8-4b04-a8c0-1868cf2be93c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.911560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.912174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.912456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.912728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.913415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:14.913730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.915323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.931455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1247/2501] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:14.932451 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:14.943494 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b61f58-2509-42c8-a80f-cb9d499aefd5 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:14.949339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:14.950374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:14.950654 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:14.950924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:14.952934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:14.954864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:14.989453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1255/2502] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:14 2022] POST /identity/v3/projects => generated 359 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:14.990589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.001958 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861160f-30f8-4db2-9b30-453170841d04 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.081301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.082299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.082742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.083170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.084016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.086125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.135725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1248/2503] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:14 2022] POST /identity/v3/users => generated 280 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:15.139339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.154174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a8a9-5587-4dbe-8a42-c62d05888715 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.165180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.165943 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.166321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.166683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.167433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.169366 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.183322 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1256/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:15.187692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.206225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5010-10fc-4bd0-873d-2e3f7f346d16 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.213727 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.214442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.214442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.215001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.215954 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.217973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a70d-c372-4440-beeb-c59b9f035d55 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.230746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1249/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:15.234753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.246012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0d026-a31a-45f2-8688-f4274c02601c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.253150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.253885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.254343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.254641 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] PATH_INFO: `/v3/projects/17cbe41ffe284f4f99494b0af23bdaea/users/c8314044d5c0469fb9fc2386ace1ee44/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.272553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cbe41ffe284f4f99494b0af23bdaea, user_id=c8314044d5c0469fb9fc2386ace1ee44, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.275891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.307355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.307752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.308023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.310900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1257/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:15 2022] PUT /identity/v3/projects/17cbe41ffe284f4f99494b0af23bdaea/users/c8314044d5c0469fb9fc2386ace1ee44/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:15.337676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] MFA Rules not processed for user `c8314044d5c0469fb9fc2386ace1ee44`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:15.338874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:15.401755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:15.410560 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.410966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1250/2507] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:15.475075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fad7-45df-4eef-8bf5-d4fb0d243434 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.479619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.480214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.480499 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.480767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.481422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.483295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.510115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1258/2508] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:15 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:15.514123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.525480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5980c-363c-4b9f-9538-e0e6ee021289 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.592345 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.593255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.593637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.594034 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.594694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.597223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.625233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1251/2509] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:15 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:15.630343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.640866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca435-8c57-4d8a-bf5e-8b165259a6e2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.645609 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.646292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.646574 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.646847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.647486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.649238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.663545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1259/2510] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:15.666175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.677897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2a3eb-6573-4896-9d5b-c4519a4168d2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.682682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.683487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.683603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.684053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] PATH_INFO: `/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e/users/7b2edcdaf2ee4fb8843d38384db4e0e1/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.702789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd45f5b9030e41b8aeb55f0c549c4b7e, user_id=7b2edcdaf2ee4fb8843d38384db4e0e1, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.705377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.720707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1252/2511] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:15 2022] PUT /identity/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e/users/7b2edcdaf2ee4fb8843d38384db4e0e1/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:15.725287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.735578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d92c0-9cb8-4c13-867d-5b4a2a8af3a3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.801014 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.801624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.801904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.802178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.802849 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.804437 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f039e-1453-4814-88a8-b9761215fd9f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.814556 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1260/2512] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:15.819268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.829622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55513d52-7548-4a9f-962a-a923365e1c4e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:15.835218 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:15.835871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.836144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.836411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] PATH_INFO: `/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e/users/7b2edcdaf2ee4fb8843d38384db4e0e1/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.838385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd45f5b9030e41b8aeb55f0c549c4b7e, user_id=7b2edcdaf2ee4fb8843d38384db4e0e1, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:15.841023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:15.851967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1253/2513] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:15 2022] PUT /identity/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e/users/7b2edcdaf2ee4fb8843d38384db4e0e1/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:15.862103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:15.862442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:15.862762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:15.889550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:15.892559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] MFA Rules not processed for user `7b2edcdaf2ee4fb8843d38384db4e0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:15.894571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:15.970799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1261/2514] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:15.982521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:15.984191 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:16.051285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d00a-4472-4287-9bea-da76f40de929 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:16.056933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:16.057611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:16.057890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:16.058166 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:16.072567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:16.074096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:16.087360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1254/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:16.178013 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:16.190935 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6295ed-edf1-4c56-bca5-07ebaf06e0eb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:16.196010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:16.196553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:16.196833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:16.197243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:16.273440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:16.274795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c5757-0b8f-4881-9cbc-e0a196397f88 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:16.284139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1262/2516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:19.033500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:19.090063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:19.097029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d752469-9bda-472c-9edb-5be1fd1fb478 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:19.102114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:19.102766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:19.103042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:19.103309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] PATH_INFO: `/v3/users/c8314044d5c0469fb9fc2386ace1ee44` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:19.104442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8314044d5c0469fb9fc2386ace1ee44)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:19.106376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:19.154505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8314044d5c0469fb9fc2386ace1ee44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.155202 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8314044d5c0469fb9fc2386ace1ee44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.155806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8314044d5c0469fb9fc2386ace1ee44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.162875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8314044d5c0469fb9fc2386ace1ee44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.176237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8314044d5c0469fb9fc2386ace1ee44'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.190005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1255/2517] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:19 2022] DELETE /identity/v3/users/c8314044d5c0469fb9fc2386ace1ee44 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:19.195432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:19.195946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:19.196059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:19.215976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:19.216868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:19.276338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1263/2518] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:19.286093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:19.287462 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:19.364871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664eea-a3c0-4f19-85b3-4ace1be250b6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:19.368525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:19.369217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:19.369510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:19.369802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:19.379771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:19.381233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:19.390488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1256/2519] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:19.657600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:19.659809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:19.672767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc30657-b990-4035-a7e6-b698d9f68133 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:19.679351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:19.679977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:19.680247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:19.680511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] PATH_INFO: `/v3/projects/17cbe41ffe284f4f99494b0af23bdaea` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:19.681633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=17cbe41ffe284f4f99494b0af23bdaea)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:19.683519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:19.722076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:19.734587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17cbe41ffe284f4f99494b0af23bdaea', 'request_id': 'req-ebfe2696-b0f0-4c12-a155-54f401246caf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.740911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17cbe41ffe284f4f99494b0af23bdaea', 'request_id': 'req-ebfe2696-b0f0-4c12-a155-54f401246caf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.747795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17cbe41ffe284f4f99494b0af23bdaea', 'request_id': 'req-ebfe2696-b0f0-4c12-a155-54f401246caf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.758913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] The token cache is being invalidate because project 17cbe41ffe284f4f99494b0af23bdaea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:19.759193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.759799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.761356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1264/2520] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:19 2022] DELETE /identity/v3/projects/17cbe41ffe284f4f99494b0af23bdaea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:19.765389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:19.775390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13a073-fa2f-4413-b8e3-8ab486a7c06f service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:19.825379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:19.832635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:19.833458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:19.833746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:19.834175 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] PATH_INFO: `/v3/users/7b2edcdaf2ee4fb8843d38384db4e0e1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:19.835757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2edcdaf2ee4fb8843d38384db4e0e1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:19.838169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:19.883146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2edcdaf2ee4fb8843d38384db4e0e1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.883789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2edcdaf2ee4fb8843d38384db4e0e1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.884405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2edcdaf2ee4fb8843d38384db4e0e1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.891220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2edcdaf2ee4fb8843d38384db4e0e1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.903874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2edcdaf2ee4fb8843d38384db4e0e1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:19.917265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1257/2521] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:19 2022] DELETE /identity/v3/users/7b2edcdaf2ee4fb8843d38384db4e0e1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:20.114750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.116140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.173298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe2696-b0f0-4c12-a155-54f401246caf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.199622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.202024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.228119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.228734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.229041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.229313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.231056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.296899 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9677f1-2856-4ac6-8c58-cd121af7e6ee admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.298345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.299966 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c127c-a7c8-4d36-b182-83eec034b79d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.301752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.302379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.302657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.303032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] PATH_INFO: `/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.304154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd45f5b9030e41b8aeb55f0c549c4b7e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.306100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.309929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1265/2522] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:20.353747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd45f5b9030e41b8aeb55f0c549c4b7e', 'request_id': 'req-794a1270-6c6b-460c-afd6-4f7a0c555eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:20.359932 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd45f5b9030e41b8aeb55f0c549c4b7e', 'request_id': 'req-794a1270-6c6b-460c-afd6-4f7a0c555eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:20.367703 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd45f5b9030e41b8aeb55f0c549c4b7e', 'request_id': 'req-794a1270-6c6b-460c-afd6-4f7a0c555eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:20.380236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] The token cache is being invalidate because project cd45f5b9030e41b8aeb55f0c549c4b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:20.380549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:20.381215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-794a1270-6c6b-460c-afd6-4f7a0c555eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:20.382735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1258/2523] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:20 2022] DELETE /identity/v3/projects/cd45f5b9030e41b8aeb55f0c549c4b7e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:20.387438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.387755 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.388043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.407871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:20.409279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.431676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:20.475635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1266/2524] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:20.484545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.484833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.485198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.499990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:20.507400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:20.509389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.518144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1259/2525] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:20.523053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.532485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a09e-7e82-4e7f-9b02-cfd20aea46ac None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.537304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.537902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.538178 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.538446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.539084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:20.539384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.540985 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.557086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1267/2526] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:20.558355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.568894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad626-f6b7-4883-afef-c2d4219e4af1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.573618 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.574244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.574521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.574797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.575428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.577381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.607797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1260/2527] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:20.612563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.622587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4a26-2807-4298-8f1e-63409bc76799 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.681285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.681899 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.682175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.682443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.683089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.685020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.710086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1268/2528] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:20.714228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.725020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cc51f-dbdb-466b-b2c4-75f9496d5f7d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.729673 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.730301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.730598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.730850 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.731489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.733125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.745823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1261/2529] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:32:20.748603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.760137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9e2-a652-4bda-b488-5b9e7dc78bda admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.764822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.765506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.765781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.766050 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.766715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.768313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019d83-1995-414f-908b-1de6cf51ecd3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.779237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1269/2530] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:20.783536 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:20.793538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492ace-dc95-43a8-a53c-b097e142fe2c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:20.798359 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:20.798789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.799071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.799847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] PATH_INFO: `/v3/projects/e4318bef1c034ddcb11beeee761402e5/users/a1ca8d88efbe494280edabf51873bb94/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.814806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4318bef1c034ddcb11beeee761402e5, user_id=a1ca8d88efbe494280edabf51873bb94, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:20.818130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:20.832963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1262/2531] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:20 2022] PUT /identity/v3/projects/e4318bef1c034ddcb11beeee761402e5/users/a1ca8d88efbe494280edabf51873bb94/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:20.844481 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:20.845021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:20.845303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:20.873329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] MFA Rules not processed for user `a1ca8d88efbe494280edabf51873bb94`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:20.874625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:20.942529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1270/2532] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:20.947147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.019306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec215e-7ae3-4d91-95a1-e80081502efa admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.024079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.024686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.024983 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.025258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.025913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.027806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.057837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1263/2533] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:20 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:21.062251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.073105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0c3ae-f2b1-42e8-8acb-a2a2108a2feb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.092223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:21.135925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.136514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.136787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.137083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.137719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.139611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.162517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1271/2534] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:21 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:21.166633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.180433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103bf-ddd3-41a3-83ae-1e150a149b78 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.185377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.186055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.186598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.186869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.187513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.189159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.200002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1264/2535] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:32:21.205138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.218426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb89b-76ae-4c8f-93c7-4b7a0c7bc5a2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.224184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.224857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.225166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.225438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] PATH_INFO: `/v3/projects/35f6d92665ea4e368e5b6b9cba998489/users/e88c8c7507bc446c99873580e32ad357/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.243510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f6d92665ea4e368e5b6b9cba998489, user_id=e88c8c7507bc446c99873580e32ad357, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.246210 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.264209 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1272/2536] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:21 2022] PUT /identity/v3/projects/35f6d92665ea4e368e5b6b9cba998489/users/e88c8c7507bc446c99873580e32ad357/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:21.268242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.280114 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0949555-b175-4831-a98e-90024afcfc04 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.345238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.345857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.346803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.347098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.347943 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.349632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad558-c98f-4ff1-8d21-1c21316e1da0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.363537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1265/2537] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:21.364706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.375396 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325b5e-6503-4b7c-bdbf-3f48c2d4e756 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.381106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.381783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.382141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.382485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] PATH_INFO: `/v3/projects/35f6d92665ea4e368e5b6b9cba998489/users/e88c8c7507bc446c99873580e32ad357/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.385386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f6d92665ea4e368e5b6b9cba998489, user_id=e88c8c7507bc446c99873580e32ad357, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.388124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.404984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1273/2538] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:21 2022] PUT /identity/v3/projects/35f6d92665ea4e368e5b6b9cba998489/users/e88c8c7507bc446c99873580e32ad357/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:21.410969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.411253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.411538 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.437869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:21.441160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] MFA Rules not processed for user `e88c8c7507bc446c99873580e32ad357`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:21.443154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:21.512902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1266/2539] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:21.524425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:21.526216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:21.585158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a26f4-d910-444a-a98b-3ce33950b678 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:21.588797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:21.589512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:21.589728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:21.589994 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:21.649855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:21.651300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:21.661309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1274/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:23.554486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:23.555980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:23.621379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f736e-46e0-4a9e-9478-894b311a731a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:23.625891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:23.626550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:23.626828 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:23.627098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] PATH_INFO: `/v3/users/764cfe23613e4a97ba05f8bf0ed77ef1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:23.628889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764cfe23613e4a97ba05f8bf0ed77ef1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:23.630860 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:23.677104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764cfe23613e4a97ba05f8bf0ed77ef1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:23.677747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764cfe23613e4a97ba05f8bf0ed77ef1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:23.678339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764cfe23613e4a97ba05f8bf0ed77ef1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:23.685544 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764cfe23613e4a97ba05f8bf0ed77ef1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:23.698563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-17e3f948-4585-47ed-b429-67f7477ebf66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764cfe23613e4a97ba05f8bf0ed77ef1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:23.711739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1267/2541] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:23 2022] DELETE /identity/v3/users/764cfe23613e4a97ba05f8bf0ed77ef1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:23.885551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:23.905024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed586e-da6a-4161-b02c-f3c03bc2db2c service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:23.931955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:23.963652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:23.964263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:23.964537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:23.964804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] PATH_INFO: `/v3/projects/60646ef7a4ad4eb1860952504092742d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:23.965923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60646ef7a4ad4eb1860952504092742d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:23.967794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.012497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60646ef7a4ad4eb1860952504092742d', 'request_id': 'req-678b847a-3e10-4ede-b88f-abe062719565'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:24.017826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60646ef7a4ad4eb1860952504092742d', 'request_id': 'req-678b847a-3e10-4ede-b88f-abe062719565'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:24.025769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60646ef7a4ad4eb1860952504092742d', 'request_id': 'req-678b847a-3e10-4ede-b88f-abe062719565'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:24.037416 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] The token cache is being invalidate because project 60646ef7a4ad4eb1860952504092742d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:24.037699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:24.038324 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-678b847a-3e10-4ede-b88f-abe062719565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:24.039807 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1275/2542] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:23 2022] DELETE /identity/v3/projects/60646ef7a4ad4eb1860952504092742d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:24.044621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.044904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.045229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.065341 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:24.066988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:24.079300 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:24.131119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1268/2543] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:24.138864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.139146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.139419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.159444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:24.160646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:24.168619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1276/2544] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:24.172551 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.182270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca153da7-564a-4105-a96f-dcae72b8814b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.187103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.187691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.187966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.188235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.188877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:24.189213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.191484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.202845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1269/2545] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:24.207829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.216907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc6b1b-75f4-4178-b79d-59c4cfdc4720 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.221557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.222147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.222422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.222693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.223331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.225250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.251088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1277/2546] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:24.255952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.265982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad5876-8761-4a66-a407-003fcae5e3c7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.323201 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.323796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.324072 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.324342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.324998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.326908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.348133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1270/2547] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:24.352297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.363423 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541837-df56-49ee-a4c0-24d4fe40cd56 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.368228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.368833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.369150 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.369424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.370071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.371670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.384493 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1278/2548] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:24.386824 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.396986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeff0d-9a1d-4c8d-9585-a7a995717a1c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.401630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.402215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.402491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.402763 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.403404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.405033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16984-1e54-49b4-9930-e918e327069d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.416466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1271/2549] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:24.421922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.433062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2195a-2a3d-45ec-bdf8-3da2a677ff76 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.437877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.438516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.438797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.439069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] PATH_INFO: `/v3/projects/e829547ae264479b98ca0701de58af02/users/aae68310c1c142a0af9920da2e1378e5/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.453812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e829547ae264479b98ca0701de58af02, user_id=aae68310c1c142a0af9920da2e1378e5, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.456779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.469071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1279/2550] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:24 2022] PUT /identity/v3/projects/e829547ae264479b98ca0701de58af02/users/aae68310c1c142a0af9920da2e1378e5/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:24.481316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.481626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.481903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.510493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] MFA Rules not processed for user `aae68310c1c142a0af9920da2e1378e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:24.511699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:24.578314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1272/2551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:24.583327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.639784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:24.645806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dccea-da6f-41b5-9870-51dfd4720d02 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.651561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.652165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.652446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.652720 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.653395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.655288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.682418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1280/2552] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:24.687139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.702801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec1fb8-55ef-4ad6-b120-8da3e7e1ca36 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.711547 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:24.759819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.760399 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.760676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.760963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.761595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.763489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.784825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1273/2553] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:24 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:24.788820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.798889 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915f2d1-9e93-461e-8fa6-4206240a2225 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.804088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.804672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.804969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.805238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.805863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.807432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.817923 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1281/2554] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:24.821837 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.831710 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a415f-1a4b-4a67-b641-a517df4dfb75 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.836315 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.836972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.837247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.837517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] PATH_INFO: `/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a/users/5efe465b4da6441bb4cfc016fd995466/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.852877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc31e9fdcd84b3eaae8348caf38655a, user_id=5efe465b4da6441bb4cfc016fd995466, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.855562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.868557 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1274/2555] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:24 2022] PUT /identity/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a/users/5efe465b4da6441bb4cfc016fd995466/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:24.872660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.883868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65e93a-9e5c-46df-84d9-1e31b9ba3084 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:24.955284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:24.956462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:24.956462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:24.956462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:24.958905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:24.960514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44516b-5d86-4a22-9ac2-f9d9b9290c5d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:24.972806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1282/2556] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:32:24.984858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:24.997054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd56f8-c175-451a-ac06-743bfd4896d3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:25.001973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:25.003015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:25.003286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:25.003554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] PATH_INFO: `/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a/users/5efe465b4da6441bb4cfc016fd995466/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:25.005846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc31e9fdcd84b3eaae8348caf38655a, user_id=5efe465b4da6441bb4cfc016fd995466, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:25.008439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:25.018916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1275/2557] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:24 2022] PUT /identity/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a/users/5efe465b4da6441bb4cfc016fd995466/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:25.029144 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:25.029446 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:25.029715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:25.057853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] MFA Rules not processed for user `5efe465b4da6441bb4cfc016fd995466`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:25.059032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:25.124730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1283/2558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:25.135478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:25.137250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:25.218775 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac23ea-6224-4fcc-9fd8-38ad754e28b3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:25.223333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:25.223930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:25.224205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:25.224473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:25.233317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:25.234725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368644-1e5c-41cc-b0a8-7cb00d5b7e74 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:25.244165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1276/2559] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:26.579357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:26.631098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:26.644237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6749a5-bd8e-4c9a-96ce-a972d5f89ed1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:26.648863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:26.649528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:26.649800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:26.650068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] PATH_INFO: `/v3/users/aae68310c1c142a0af9920da2e1378e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:26.651269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae68310c1c142a0af9920da2e1378e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:26.653211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:26.699165 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae68310c1c142a0af9920da2e1378e5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:26.699739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae68310c1c142a0af9920da2e1378e5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:26.707598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae68310c1c142a0af9920da2e1378e5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:26.708225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae68310c1c142a0af9920da2e1378e5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:26.720115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae68310c1c142a0af9920da2e1378e5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:26.732618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1284/2560] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:26 2022] DELETE /identity/v3/users/aae68310c1c142a0af9920da2e1378e5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:26.737208 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:26.737833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:26.737833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:26.756401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:26.757715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:26.801474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:26.817659 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1277/2561] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:26.827188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:26.828608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:26.886743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066fe84-b010-404c-81a1-2b2044d08f7c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:26.890362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:26.890954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:26.891225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:26.891488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:26.901424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:26.902832 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:26.912653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1285/2562] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:27.181875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:27.183325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:27.200996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156821cd-160e-43f8-9f8d-31cc8b35a523 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:27.206246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:27.206880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.207160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.207405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] PATH_INFO: `/v3/projects/e829547ae264479b98ca0701de58af02` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.208518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e829547ae264479b98ca0701de58af02)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:27.210489 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:27.255813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e829547ae264479b98ca0701de58af02', 'request_id': 'req-28f8f1b6-80f7-42fd-b382-4639d408dbdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.261219 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e829547ae264479b98ca0701de58af02', 'request_id': 'req-28f8f1b6-80f7-42fd-b382-4639d408dbdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.269348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e829547ae264479b98ca0701de58af02', 'request_id': 'req-28f8f1b6-80f7-42fd-b382-4639d408dbdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.280243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] The token cache is being invalidate because project e829547ae264479b98ca0701de58af02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:27.280529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.281174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.282645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1278/2563] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:27 2022] DELETE /identity/v3/projects/e829547ae264479b98ca0701de58af02 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:27.286635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:27.296642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01970de9-9d85-412c-a42c-10819772daf8 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:27.351269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:27.351880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.352153 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.352422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] PATH_INFO: `/v3/users/5efe465b4da6441bb4cfc016fd995466` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.353528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5efe465b4da6441bb4cfc016fd995466)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:27.355408 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:27.368478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:27.400159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5efe465b4da6441bb4cfc016fd995466'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.400798 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5efe465b4da6441bb4cfc016fd995466'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.407475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5efe465b4da6441bb4cfc016fd995466'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.408089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5efe465b4da6441bb4cfc016fd995466'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.420476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5efe465b4da6441bb4cfc016fd995466'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.434066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1286/2564] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:27 2022] DELETE /identity/v3/users/5efe465b4da6441bb4cfc016fd995466 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:27.484723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:27.486160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:27.546617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f1b6-80f7-42fd-b382-4639d408dbdf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:27.597901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:27.598513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.598795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.599067 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.600776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:27.645602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:27.667723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:27.669206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb23552-055f-45e3-a710-3d27e6276ff1 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:27.679225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1279/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:27.707454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:27.708934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:27.780325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be331-d8b9-49ba-beaf-74a8ef7fec9a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:27.785893 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:27.786512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.786785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.787049 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] PATH_INFO: `/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.788169 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc31e9fdcd84b3eaae8348caf38655a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:27.790066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:27.840029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc31e9fdcd84b3eaae8348caf38655a', 'request_id': 'req-9fd14ebe-86b7-4581-8813-9884cccd0dae'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.847836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc31e9fdcd84b3eaae8348caf38655a', 'request_id': 'req-9fd14ebe-86b7-4581-8813-9884cccd0dae'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.855216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc31e9fdcd84b3eaae8348caf38655a', 'request_id': 'req-9fd14ebe-86b7-4581-8813-9884cccd0dae'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.870546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] The token cache is being invalidate because project dfc31e9fdcd84b3eaae8348caf38655a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:27.870679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.871325 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fd14ebe-86b7-4581-8813-9884cccd0dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:27.872829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1287/2566] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:27 2022] DELETE /identity/v3/projects/dfc31e9fdcd84b3eaae8348caf38655a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:27.877749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.878310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.878539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.898505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:27.899705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:27.961788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1280/2567] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:27.969558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:27.969842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:27.970110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:27.993424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:27.995510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:28.008724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1288/2568] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:28.010642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.020982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eca8f-7017-455e-aeef-aa5905f05cf1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.030667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.030667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.030667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.030667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.031060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:28.031060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.031060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.043854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1281/2569] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:28.049004 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.059135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ee0ee-d4e4-446d-b83e-735d02f02573 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.064068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.064664 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.065076 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.065548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.066275 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.068180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.083678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:28.098106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1289/2570] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:28.102226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.112303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812add1-9049-415c-b37c-2f5063be4ccc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.167779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.168376 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.168652 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.168924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.169593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.171496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.192967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1282/2571] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:28.197093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.207579 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1422b-9962-489d-9e5c-9463df2e215f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.212097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.212679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.212986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.213246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.213900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.215465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.225670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1290/2572] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:28.229445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.239589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7ae3-201a-4957-b6a0-7e1abc01b12b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.244193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.244771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.245073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.245347 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.245984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.247550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33655a08-c7e0-4d34-bb35-e70a6be2a918 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.259276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1283/2573] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:28.263299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.273697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca82e-2be7-4371-b507-d1f80d1489e1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.278235 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.278865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.279136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.279399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] PATH_INFO: `/v3/projects/12b0731570c64542baca085ef1bb3cde/users/2bc689d237154ecd8ecf831d1d0f4180/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.295482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b0731570c64542baca085ef1bb3cde, user_id=2bc689d237154ecd8ecf831d1d0f4180, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.298056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.310410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1291/2574] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:28 2022] PUT /identity/v3/projects/12b0731570c64542baca085ef1bb3cde/users/2bc689d237154ecd8ecf831d1d0f4180/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:28.321933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.322221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.322495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.350534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] MFA Rules not processed for user `2bc689d237154ecd8ecf831d1d0f4180`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:28.351743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:28.383822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:28.417655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1284/2575] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:28.422633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.482719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598578a-ef5d-44ef-b821-4c211bda7b26 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.487598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.488186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.488461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.488728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.489399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.491270 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.517485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1292/2576] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:28.522099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.532939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a508d-2d59-494b-a46d-82b8bfcce9a1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.587957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.588491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.588765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.589065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.589694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.591584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.611625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1285/2577] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:28.615625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.625749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1daa19-0fa9-4b13-af8a-4b923239bfba admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.631023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.631616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.631917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.632181 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.632813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.634450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.645536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1293/2578] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:28.649435 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.659221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c70a-32a6-4afa-b9ba-fa406e4dac9b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.663770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.664402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.664674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.664962 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] PATH_INFO: `/v3/projects/c6dfad34c73347c5ae974d146c38be02/users/d7c1ddfb680844d0b9f4470f7e4119fd/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.679662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dfad34c73347c5ae974d146c38be02, user_id=d7c1ddfb680844d0b9f4470f7e4119fd, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.682372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.695999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1286/2579] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:28 2022] PUT /identity/v3/projects/c6dfad34c73347c5ae974d146c38be02/users/d7c1ddfb680844d0b9f4470f7e4119fd/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:28.699846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.710443 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0fc7-0805-4f8d-a3db-2f6e0861ffc6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.763922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.764507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.764788 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.765084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.765714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.767286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a2e3-8650-40ed-99e5-115e9927a080 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.777175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1294/2580] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:28.781327 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.791743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a518b9-f0c6-47f4-9e24-6cb472480d57 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.796288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.796918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.797220 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.797488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] PATH_INFO: `/v3/projects/c6dfad34c73347c5ae974d146c38be02/users/d7c1ddfb680844d0b9f4470f7e4119fd/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.799998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dfad34c73347c5ae974d146c38be02, user_id=d7c1ddfb680844d0b9f4470f7e4119fd, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:28.802639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:28.812799 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1287/2581] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:28 2022] PUT /identity/v3/projects/c6dfad34c73347c5ae974d146c38be02/users/d7c1ddfb680844d0b9f4470f7e4119fd/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:28.823333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.823619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.823892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:28.850606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] MFA Rules not processed for user `d7c1ddfb680844d0b9f4470f7e4119fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:28.851708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:28.868479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:28.921650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1295/2582] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:28.933227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:28.934913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:28.991042 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17f69-c970-4ee4-85d6-46c6f11f0287 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:28.994617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:28.995200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:28.995473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:28.995740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:29.005514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:29.006930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:29.016157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1288/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:29.117652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:29.119032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:29.131495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d2264-efcf-46a3-af00-c963c7b7ade9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:29.135018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:29.135608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:29.135879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:29.136142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:29.190718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:29.192121 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77651-ce0b-4c24-9c12-41805a1913a7 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:29.200747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1296/2584] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:32.834877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:32.865986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:32.899306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4c9f6-b337-4b38-83c2-80d312f92ec9 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:32.903888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:32.904519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:32.904796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:32.905103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] PATH_INFO: `/v3/users/2bc689d237154ecd8ecf831d1d0f4180` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:32.906230 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc689d237154ecd8ecf831d1d0f4180)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:32.908130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:32.956789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc689d237154ecd8ecf831d1d0f4180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:32.957368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc689d237154ecd8ecf831d1d0f4180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:32.957960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc689d237154ecd8ecf831d1d0f4180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:32.964761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc689d237154ecd8ecf831d1d0f4180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:32.976697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc689d237154ecd8ecf831d1d0f4180'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:32.989646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1289/2585] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:32 2022] DELETE /identity/v3/users/2bc689d237154ecd8ecf831d1d0f4180 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:32.994317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:32.994801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:32.995156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:33.014512 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:33.015699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:33.073040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1297/2586] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:33.083439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:33.084839 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:33.139381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831b41-313e-41d8-9360-b553d290bbfc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:33.143609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:33.144203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:33.144469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:33.144733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:33.153449 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:33.154849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:33.164131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1290/2587] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:33.427194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:33.428909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:33.442387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b266a78-644f-45ab-898f-ac97ba5b9e27 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:33.454887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:33.456175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:33.456886 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:33.457376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] PATH_INFO: `/v3/projects/12b0731570c64542baca085ef1bb3cde` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:33.458668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b0731570c64542baca085ef1bb3cde)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:33.460828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:33.513861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:33.515771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b0731570c64542baca085ef1bb3cde', 'request_id': 'req-4033021f-9d5e-4fee-a624-a844ae7f1a41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.523332 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b0731570c64542baca085ef1bb3cde', 'request_id': 'req-4033021f-9d5e-4fee-a624-a844ae7f1a41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.534198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b0731570c64542baca085ef1bb3cde', 'request_id': 'req-4033021f-9d5e-4fee-a624-a844ae7f1a41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.547126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] The token cache is being invalidate because project 12b0731570c64542baca085ef1bb3cde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:33.547419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.548221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.551474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1298/2588] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:33 2022] DELETE /identity/v3/projects/12b0731570c64542baca085ef1bb3cde => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:33.555379 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:33.575335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f444147-4d7f-462e-84cc-69fb79e6fe1d service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:33.633921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:33.646177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:33.646852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:33.647129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:33.647413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] PATH_INFO: `/v3/users/d7c1ddfb680844d0b9f4470f7e4119fd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:33.648510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c1ddfb680844d0b9f4470f7e4119fd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:33.650454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:33.698464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c1ddfb680844d0b9f4470f7e4119fd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.699133 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c1ddfb680844d0b9f4470f7e4119fd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.699751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c1ddfb680844d0b9f4470f7e4119fd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.708532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c1ddfb680844d0b9f4470f7e4119fd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.724702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b4439584-747c-465e-bd68-49c1d079c335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c1ddfb680844d0b9f4470f7e4119fd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:33.737503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1291/2589] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:33 2022] DELETE /identity/v3/users/d7c1ddfb680844d0b9f4470f7e4119fd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:34.015453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.017182 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:34.033509 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033021f-9d5e-4fee-a624-a844ae7f1a41 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.090411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.091044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.091326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.091595 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] PATH_INFO: `/v3/projects/c6dfad34c73347c5ae974d146c38be02` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.092713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6dfad34c73347c5ae974d146c38be02)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.094625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.140572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6dfad34c73347c5ae974d146c38be02', 'request_id': 'req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:34.146708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6dfad34c73347c5ae974d146c38be02', 'request_id': 'req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:34.154523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6dfad34c73347c5ae974d146c38be02', 'request_id': 'req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:34.166363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] The token cache is being invalidate because project c6dfad34c73347c5ae974d146c38be02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:34.166653 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:34.167283 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-545db7cd-50ee-4f67-b8a3-4c1834a02fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:34.168742 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1299/2590] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:34 2022] DELETE /identity/v3/projects/c6dfad34c73347c5ae974d146c38be02 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:34.173792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.174286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.174425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.193941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:34.195146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:34.252711 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1292/2591] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:34.260743 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.261003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.261287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.282631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:34.284441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:34.292364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1300/2592] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:32:34.296530 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.306266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0af4-11c0-4c5c-b952-e0cd3cd316c1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.310782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.311366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.311893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.312159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.312802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:34.313130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.314699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.325748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1293/2593] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:34.329808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.339431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8d415-00d1-4892-bdc0-7dd5dbadc10f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.344101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.344690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.344990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.345267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.345904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.348420 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.374237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1301/2594] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:34.378573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.390033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f32c-da8c-4c5c-9060-84127ca5218a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.442236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:34.448535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.449164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.449777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.450047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.450655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.452767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.474515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1294/2595] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:34.478389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.488466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80385959-8275-497e-94ed-feaed747e45b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.492971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.493564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.493836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.494101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.494725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.496584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.507365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1302/2596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:34.511301 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.521578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beb5b6-2bca-44cb-91fd-272946a600ee admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.526426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.527027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.527302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.527574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.528238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.529880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d88c58-22c0-41fa-a3a4-ef2426a088a6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.541378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1295/2597] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:34.545448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.555205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe023b-d595-4328-ab05-0bc250dc5434 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.559804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.560439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.560714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.561016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] PATH_INFO: `/v3/projects/89424102000c43a5ab7cfda3948957de/users/0e69f7fc9d32485b8199864b146b76a2/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.575779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89424102000c43a5ab7cfda3948957de, user_id=0e69f7fc9d32485b8199864b146b76a2, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.578485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.591738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1303/2598] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:34 2022] PUT /identity/v3/projects/89424102000c43a5ab7cfda3948957de/users/0e69f7fc9d32485b8199864b146b76a2/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:34.603317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.603988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.603988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.631988 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] MFA Rules not processed for user `0e69f7fc9d32485b8199864b146b76a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:34.633245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:34.695367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1296/2599] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:34.699855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.704775 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:34.764514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4494379-a41b-42fc-9c19-e6426518c208 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.769770 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.770376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.770652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.770922 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.771566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.773496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.800913 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1304/2600] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:34.805005 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.816568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cceb52-3714-454b-9bec-ec6c76286bef None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.871221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.871819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.872096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.872366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.873016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.874923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.896023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1297/2601] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:34 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:34.900376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.910897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22f49-306a-4696-a1e2-5b9cb1118d52 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.915448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.916033 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.916305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.916568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.917309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.918875 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.929939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1305/2602] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:32:34.934050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.945639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b5db8-ce10-436a-97de-b955b916c148 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.950177 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.950762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.951035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.951303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.951939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.953552 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52774ea-6521-4255-baeb-a8befc1eb65f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:34.963270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1298/2603] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:34.967145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:34.976671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc86d7-7793-43fd-8831-1bd37249b514 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:34.981198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:34.981830 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:34.982101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:34.982363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] PATH_INFO: `/v3/projects/84ac1b8fd6aa465b9d66a62043e5cfce/users/1aec40a72b804d0eb52fe77b3eabdc3f/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:34.996917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ac1b8fd6aa465b9d66a62043e5cfce, user_id=1aec40a72b804d0eb52fe77b3eabdc3f, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:34.999533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.012619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1306/2604] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:34 2022] PUT /identity/v3/projects/84ac1b8fd6aa465b9d66a62043e5cfce/users/1aec40a72b804d0eb52fe77b3eabdc3f/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:35.022916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.023141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.023412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.051354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] MFA Rules not processed for user `1aec40a72b804d0eb52fe77b3eabdc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:35.052560 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:35.086424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:35.119483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1299/2605] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:35.124045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.185495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff4a8-58b0-49e9-88f0-32fdf93587c4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.190878 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.191467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.191753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.192021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.192644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.194553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.220651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1307/2606] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:35 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:35.224717 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.235131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e8a0-2a13-43ec-b3af-134a007ee474 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.289783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.290378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.290654 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.290923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.291541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.293465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.315337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1300/2607] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:35 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:35.319305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.329206 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb22e6-67af-4d3b-8a90-58ff93f6ada3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.334592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.335190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.335467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.335734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.336384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.338035 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.348728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1308/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:35.352690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.362777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb264dd-90bb-4b7a-92d2-fa747e20b233 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.367333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.367964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.368237 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.368507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] PATH_INFO: `/v3/projects/a537315e54f8458884130ff19c794e3a/users/6bff642b1c3e4d42a46a448c41fad018/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.383296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a537315e54f8458884130ff19c794e3a, user_id=6bff642b1c3e4d42a46a448c41fad018, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.385957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.399226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1301/2609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:35 2022] PUT /identity/v3/projects/a537315e54f8458884130ff19c794e3a/users/6bff642b1c3e4d42a46a448c41fad018/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:35.403269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.413139 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dd97d-ac9a-45e6-ac66-1564ccd2f245 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.437285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:35.470958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.471601 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.471874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.472155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.472963 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.474467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50f687-b6c5-4847-9a4b-c23599797aca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.484858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1309/2610] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:35.489105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.499378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988fb21-8e29-4157-9371-af9dd90dbab4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.504588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.505255 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.505531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.505822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] PATH_INFO: `/v3/projects/a537315e54f8458884130ff19c794e3a/users/6bff642b1c3e4d42a46a448c41fad018/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.507704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a537315e54f8458884130ff19c794e3a, user_id=6bff642b1c3e4d42a46a448c41fad018, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.510563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.521398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1302/2611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:35 2022] PUT /identity/v3/projects/a537315e54f8458884130ff19c794e3a/users/6bff642b1c3e4d42a46a448c41fad018/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:35.532495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.532781 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.533084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.560656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] MFA Rules not processed for user `6bff642b1c3e4d42a46a448c41fad018`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:35.562061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:35.629431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1310/2612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:35.640628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:35.642455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:35.700648 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec579f49-2871-4c45-a23a-29dcc9f886f2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:35.704349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:35.704966 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:35.705256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:35.706295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:35.746390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:35.764170 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:35.765626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:35.774400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1303/2613] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:37.285827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:37.295327 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf2cf-3754-4cf6-9db0-ef0c3289ff18 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:37.298954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:37.300284 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:37.300565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:37.300835 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:37.357313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:37.358733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:37.368138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1311/2614] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:37.890537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:37.891950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:37.903845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600901b2-2da1-4dcb-a3c6-f76958b06654 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:37.907601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:37.908259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:37.909306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:37.909801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:37.918708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:37.920123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011dbb6-1a6a-4c9d-8ea8-7c4353886fc2 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:37.931296 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1304/2615] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:44.980341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:45.027425 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:45.043118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dc54-bbd2-404d-8242-8ae3fec0d212 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:45.047608 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:45.048229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:45.048504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:45.048769 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] PATH_INFO: `/v3/users/0e69f7fc9d32485b8199864b146b76a2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:45.050524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e69f7fc9d32485b8199864b146b76a2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:45.052413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:45.098747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e69f7fc9d32485b8199864b146b76a2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.099681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e69f7fc9d32485b8199864b146b76a2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.106734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e69f7fc9d32485b8199864b146b76a2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.107349 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e69f7fc9d32485b8199864b146b76a2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.119724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e69f7fc9d32485b8199864b146b76a2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.133036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1312/2616] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:44 2022] DELETE /identity/v3/users/0e69f7fc9d32485b8199864b146b76a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:45.137651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:45.138155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:45.138252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:45.157446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:45.159401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:45.216767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1305/2617] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:45.227145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:45.228535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:45.285230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5d4cb-9714-41fd-b44b-40e369a0dbca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:45.288858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:45.289477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:45.289753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:45.290021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:45.298803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:45.301048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:45.310223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1313/2618] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:45.572852 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:45.574293 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:45.589589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa81dda-aa66-4343-a17d-ef5b5138461b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:45.595249 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:45.596056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:45.596810 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:45.597129 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] PATH_INFO: `/v3/projects/89424102000c43a5ab7cfda3948957de` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:45.598374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89424102000c43a5ab7cfda3948957de)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:45.600227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:45.646818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89424102000c43a5ab7cfda3948957de', 'request_id': 'req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.652173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89424102000c43a5ab7cfda3948957de', 'request_id': 'req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.660116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89424102000c43a5ab7cfda3948957de', 'request_id': 'req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.671024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] The token cache is being invalidate because project 89424102000c43a5ab7cfda3948957de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:45.671320 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.671916 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.673408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1306/2619] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:45 2022] DELETE /identity/v3/projects/89424102000c43a5ab7cfda3948957de => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:45.677671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:45.688124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bc91-f3a5-43f8-b41a-0436817ce106 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:45.749918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:45.750459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:45.750733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:45.750999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] PATH_INFO: `/v3/users/1aec40a72b804d0eb52fe77b3eabdc3f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:45.752103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aec40a72b804d0eb52fe77b3eabdc3f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:45.754757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:45.765278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:45.803645 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aec40a72b804d0eb52fe77b3eabdc3f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.804291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aec40a72b804d0eb52fe77b3eabdc3f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.811389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aec40a72b804d0eb52fe77b3eabdc3f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.812279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aec40a72b804d0eb52fe77b3eabdc3f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.824776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aec40a72b804d0eb52fe77b3eabdc3f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:45.837871 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1314/2620] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:45 2022] DELETE /identity/v3/users/1aec40a72b804d0eb52fe77b3eabdc3f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:46.104206 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:46.105639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:46.120025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395e38-4c7e-413c-a9d2-a14b2a45ac50 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:46.169705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:46.182577 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:46.183207 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.183483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.183752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] PATH_INFO: `/v3/projects/84ac1b8fd6aa465b9d66a62043e5cfce` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.184885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ac1b8fd6aa465b9d66a62043e5cfce)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:46.187053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:46.232074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ac1b8fd6aa465b9d66a62043e5cfce', 'request_id': 'req-c121d1c1-fee2-4baf-a7de-810286c986e5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.238142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ac1b8fd6aa465b9d66a62043e5cfce', 'request_id': 'req-c121d1c1-fee2-4baf-a7de-810286c986e5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.248578 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ac1b8fd6aa465b9d66a62043e5cfce', 'request_id': 'req-c121d1c1-fee2-4baf-a7de-810286c986e5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.260139 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] The token cache is being invalidate because project 84ac1b8fd6aa465b9d66a62043e5cfce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:46.260427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.261109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.262582 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1307/2621] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:46 2022] DELETE /identity/v3/projects/84ac1b8fd6aa465b9d66a62043e5cfce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:46.266492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:46.267936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:46.280468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577bdb-8a1d-4d53-926c-f0b2721a7751 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:46.336315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:46.336927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.337232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.337498 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] PATH_INFO: `/v3/users/6bff642b1c3e4d42a46a448c41fad018` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.338585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bff642b1c3e4d42a46a448c41fad018)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:46.340590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:46.386879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bff642b1c3e4d42a46a448c41fad018'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.387546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bff642b1c3e4d42a46a448c41fad018'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.394606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bff642b1c3e4d42a46a448c41fad018'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.395232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bff642b1c3e4d42a46a448c41fad018'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.406936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-727de791-d878-46bf-84f2-7f009a69304d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bff642b1c3e4d42a46a448c41fad018'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.419559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1315/2622] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:46 2022] DELETE /identity/v3/users/6bff642b1c3e4d42a46a448c41fad018 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:46.690990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:46.692457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:46.705213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121d1c1-fee2-4baf-a7de-810286c986e5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:46.765706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:46.766334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.766608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.766906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] PATH_INFO: `/v3/projects/a537315e54f8458884130ff19c794e3a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.768182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a537315e54f8458884130ff19c794e3a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:46.770193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:46.817184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a537315e54f8458884130ff19c794e3a', 'request_id': 'req-cb44ab16-5787-433f-b9fd-bdce4feca9de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.822520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a537315e54f8458884130ff19c794e3a', 'request_id': 'req-cb44ab16-5787-433f-b9fd-bdce4feca9de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.829651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a537315e54f8458884130ff19c794e3a', 'request_id': 'req-cb44ab16-5787-433f-b9fd-bdce4feca9de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.840974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] The token cache is being invalidate because project a537315e54f8458884130ff19c794e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:46.841263 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.841883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cb44ab16-5787-433f-b9fd-bdce4feca9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:46.843336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1308/2623] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:46 2022] DELETE /identity/v3/projects/a537315e54f8458884130ff19c794e3a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:46.848999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.849302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.849565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.863124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:46.869338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:46.870929 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:46.931116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1316/2624] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:46.938948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.939231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.939502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.960881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:46.962754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:46.972751 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1309/2625] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:46.977177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:46.986767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73167e0e-a3af-40a2-9be0-73421ae0f123 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:46.991261 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:46.991844 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:46.992118 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:46.992386 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:46.993058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:46.993374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:46.994934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.006309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1317/2626] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:47.010791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.016297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:47.020878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8e2c9-e176-4d4e-8dfa-8079dc6c4207 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.026917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.027681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.027959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.028282 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.029380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.031429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.058984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1310/2627] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:47 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:47.063501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.073410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e6cb1-6c1b-4b20-adba-08d862a2876b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.128622 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.129260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.129538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.129811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.130450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.132353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.153460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1318/2628] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:47 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:47.157567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.168599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1000d-4c02-4fb0-a547-198fe6575eb7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.173115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.173694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.173961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.174434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.175092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.176663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.186397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1311/2629] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:47.191263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.201031 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1ae2-9ea3-4dc6-a6dc-95ab49ba9e75 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.206136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.206718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.206992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.207257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.207937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.209562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acebc6-938f-4438-ba07-2fbe81f287ea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.219882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1319/2630] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:47.223896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.233768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e19e6-b138-4bdc-9335-089c4227bfda admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.238525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.239154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.239426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.239690 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] PATH_INFO: `/v3/projects/01b7cf3a259443a9a370b79e2bc43dcc/users/a5433685dfeb41abaf7c58758390ad9b/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.254438 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b7cf3a259443a9a370b79e2bc43dcc, user_id=a5433685dfeb41abaf7c58758390ad9b, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.257085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.270124 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1312/2631] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:47 2022] PUT /identity/v3/projects/01b7cf3a259443a9a370b79e2bc43dcc/users/a5433685dfeb41abaf7c58758390ad9b/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:47.275065 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e9c5-7ad7-486c-9191-2d4b1792ebac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.275546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e9c5-7ad7-486c-9191-2d4b1792ebac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.275649 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e9c5-7ad7-486c-9191-2d4b1792ebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.295398 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bbb4e9c5-7ad7-486c-9191-2d4b1792ebac None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:47.296572 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bbb4e9c5-7ad7-486c-9191-2d4b1792ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:47.353682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1320/2632] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:47.363433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:47.365197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:47.421143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31717688-4259-413c-9c5d-319b3c6a0c9c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:47.425361 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:47.425967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:47.426245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:47.426519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:47.435674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:47.437386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:47.447596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1313/2633] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:48.063640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d172-35d0-4516-a487-45ca209c00df None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:48.063927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d172-35d0-4516-a487-45ca209c00df None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:48.064198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d172-35d0-4516-a487-45ca209c00df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:48.090539 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:48.093749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a0d6d172-35d0-4516-a487-45ca209c00df None None] MFA Rules not processed for user `a5433685dfeb41abaf7c58758390ad9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:48.096477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a0d6d172-35d0-4516-a487-45ca209c00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:48.164441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1321/2634] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:48.176049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:48.177479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:48.233493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ff56-76c4-4bfa-bdee-bdf3b930fc9f service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:48.237661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:48.238213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:48.238486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:48.238748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:48.250413 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:48.251840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:48.261111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1314/2635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:49.619379 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a88f-cade-4f9a-a1e2-b0c460d550b3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:49.619885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a88f-cade-4f9a-a1e2-b0c460d550b3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:49.619999 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a88f-cade-4f9a-a1e2-b0c460d550b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:49.638584 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-84b7a88f-cade-4f9a-a1e2-b0c460d550b3 None None] MFA Rules not processed for user `a5433685dfeb41abaf7c58758390ad9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:49.639772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-84b7a88f-cade-4f9a-a1e2-b0c460d550b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:49.646585 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1322/2636] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:52.676696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:52.682332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:52.697949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018576d-c5e4-45d3-8b69-3f702bd9c44e service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:52.703039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:52.703671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:52.703952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:52.704225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] PATH_INFO: `/v3/users/a1ca8d88efbe494280edabf51873bb94` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:52.706147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ca8d88efbe494280edabf51873bb94)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:52.708330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:52.756473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ca8d88efbe494280edabf51873bb94'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:52.757184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ca8d88efbe494280edabf51873bb94'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:52.757809 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ca8d88efbe494280edabf51873bb94'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:52.759063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:52.768604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ca8d88efbe494280edabf51873bb94'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:52.782422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ca8d88efbe494280edabf51873bb94'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:52.796336 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1315/2637] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:52 2022] DELETE /identity/v3/users/a1ca8d88efbe494280edabf51873bb94 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:52.801431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:52.801955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:52.802064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:52.820829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:52.822056 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:52.899891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1323/2638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:52.910380 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:52.911933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:52.973865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a5e1-c978-48e8-8cb0-203dd0d139c2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:52.977593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:52.978198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:52.978479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:52.978748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:52.988993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:52.990429 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:53.000739 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1316/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:53.264628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:53.266092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:53.278967 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197acb9-468e-41cb-b243-e295da9912fa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:53.283594 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:53.284340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:53.284630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:53.284904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] PATH_INFO: `/v3/projects/e4318bef1c034ddcb11beeee761402e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:53.286079 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4318bef1c034ddcb11beeee761402e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:53.287998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:53.342758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4318bef1c034ddcb11beeee761402e5', 'request_id': 'req-390a3171-c6d8-4190-b61b-45a0e825f484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.348527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4318bef1c034ddcb11beeee761402e5', 'request_id': 'req-390a3171-c6d8-4190-b61b-45a0e825f484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.355636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4318bef1c034ddcb11beeee761402e5', 'request_id': 'req-390a3171-c6d8-4190-b61b-45a0e825f484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.366693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] The token cache is being invalidate because project e4318bef1c034ddcb11beeee761402e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:53.366982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.367611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.369124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1324/2640] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:53 2022] DELETE /identity/v3/projects/e4318bef1c034ddcb11beeee761402e5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:53.373186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:53.383586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4b77d-f219-4d65-a6f4-37b0eed5b139 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:53.440535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:53.441101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:53.441377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:53.441668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] PATH_INFO: `/v3/users/e88c8c7507bc446c99873580e32ad357` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:53.442789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e88c8c7507bc446c99873580e32ad357)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:53.444686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:53.489771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e88c8c7507bc446c99873580e32ad357'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.490418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e88c8c7507bc446c99873580e32ad357'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.491010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e88c8c7507bc446c99873580e32ad357'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.497883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e88c8c7507bc446c99873580e32ad357'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.508171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:53.510702 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-0054c050-4f65-4eb6-b8f0-41d72d86e748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e88c8c7507bc446c99873580e32ad357'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:53.524923 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1317/2641] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:53 2022] DELETE /identity/v3/users/e88c8c7507bc446c99873580e32ad357 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:53.875709 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:53.884243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:53.897028 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3171-c6d8-4190-b61b-45a0e825f484 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:53.914223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:53.956797 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:53.957365 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:53.957637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:53.957902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] PATH_INFO: `/v3/projects/35f6d92665ea4e368e5b6b9cba998489` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:53.958993 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f6d92665ea4e368e5b6b9cba998489)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:53.960870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.006833 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f6d92665ea4e368e5b6b9cba998489', 'request_id': 'req-9c428e48-3686-48d7-8e44-74bc28a5d991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:54.012133 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f6d92665ea4e368e5b6b9cba998489', 'request_id': 'req-9c428e48-3686-48d7-8e44-74bc28a5d991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:54.020044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f6d92665ea4e368e5b6b9cba998489', 'request_id': 'req-9c428e48-3686-48d7-8e44-74bc28a5d991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:54.031204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] The token cache is being invalidate because project 35f6d92665ea4e368e5b6b9cba998489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:54.031488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:54.032113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9c428e48-3686-48d7-8e44-74bc28a5d991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:54.033640 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1325/2642] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:53 2022] DELETE /identity/v3/projects/35f6d92665ea4e368e5b6b9cba998489 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:54.038622 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.039151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.039251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.059419 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:54.066030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:54.129475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1318/2643] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:54.137401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.137684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.138124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.158192 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:54.159372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:54.167044 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1326/2644] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:54.171329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.180982 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d0d6-9b10-41fc-9077-0b4c22fef16a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.185492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.186081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.186356 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.186627 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.187525 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:32:54.187827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.189440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.201235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1319/2645] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:32:54.206141 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.215480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b225-8c0c-4e4e-a628-1df193cff6cd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.220075 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.220669 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.221134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.221689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.222331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.224226 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.250976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1327/2646] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:54.255728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.265722 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587fdee-779e-4ba4-96f4-9a2a8d9a8d1b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.323025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.323614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.323889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.324153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.324774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.326687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.347020 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1320/2647] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:54.351145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.361171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a89c-79c2-4c3a-98d0-6608ef95cb67 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.366346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.366932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.367207 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.367474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.368102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.369723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.379991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1328/2648] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:54.383876 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.394709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb35648-c52c-484c-acc0-b7bd9de6b5a0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.399883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.400480 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.400752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.401058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.401745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.403329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752c37e-1f0f-4bb0-ac23-b6151ad579b0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.414194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1321/2649] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:54.418726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.430505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202f10-a5a3-447f-8c22-b6ad59816c85 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.435288 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.435938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.436218 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.436492 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] PATH_INFO: `/v3/projects/7a7ca27719b64c19ac8aaed0c1872b36/users/3a99824bba794269828769fb3ac0b32a/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.452883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7ca27719b64c19ac8aaed0c1872b36, user_id=3a99824bba794269828769fb3ac0b32a, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.455528 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.469019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1329/2650] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:54 2022] PUT /identity/v3/projects/7a7ca27719b64c19ac8aaed0c1872b36/users/3a99824bba794269828769fb3ac0b32a/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:54.479353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.479646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.479938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.508924 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] MFA Rules not processed for user `3a99824bba794269828769fb3ac0b32a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:54.510161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:54.554027 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:54.578805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1322/2651] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:54.583429 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.632297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:54.649765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ea5e-74cf-46a6-98b2-b610a28f94c5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.654363 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.655576 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.655858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.656128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.656763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.658711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.689607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1330/2652] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:32:54.693695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.707712 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef364b86-e9e0-461b-846b-788e2487dd23 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.776836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.777408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.777757 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.778026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.779258 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.781210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.808683 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1323/2653] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:32:54 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:32:54.812974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.826048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1034c-9895-419e-a819-46895f2e7300 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.832287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.832887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.833438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.833711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.834496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.836208 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.848761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1331/2654] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:54.852007 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.863953 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ddb3c-82f4-45b2-800b-09a07269442f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:54.869073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:54.869804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:54.870145 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:54.870969 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] PATH_INFO: `/v3/projects/aa3fe97a862f4be08f173ed799b81e54/users/7b914e02dad7459daeb991dfd13a8765/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:54.888880 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3fe97a862f4be08f173ed799b81e54, user_id=7b914e02dad7459daeb991dfd13a8765, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:54.891469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:54.909596 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1324/2655] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:54 2022] PUT /identity/v3/projects/aa3fe97a862f4be08f173ed799b81e54/users/7b914e02dad7459daeb991dfd13a8765/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:54.913551 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:54.929455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ea99-3d3e-415d-891a-28dcafae1ec9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:55.002464 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:55.003890 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:55.004321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:55.004779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:55.005596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:55.007337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1eba2b-d04f-4c0c-8336-d5ce213efcca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:55.017625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1332/2656] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:32:55.022929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:55.035135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682241-8b0f-437d-8653-5e0d9c351268 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:55.040116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:55.041100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:55.041477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:55.041846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] PATH_INFO: `/v3/projects/aa3fe97a862f4be08f173ed799b81e54/users/7b914e02dad7459daeb991dfd13a8765/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:55.043901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3fe97a862f4be08f173ed799b81e54, user_id=7b914e02dad7459daeb991dfd13a8765, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:55.046678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:55.058908 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1325/2657] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:32:55 2022] PUT /identity/v3/projects/aa3fe97a862f4be08f173ed799b81e54/users/7b914e02dad7459daeb991dfd13a8765/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:55.073276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:55.073561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:55.073851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:55.101763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] MFA Rules not processed for user `7b914e02dad7459daeb991dfd13a8765`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:55.102968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:55.172456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1333/2658] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:55.184439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:55.186235 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:55.244120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8c3a6-564e-42c5-9831-f73a93daf40e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:55.247723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:55.248330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:55.248603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:55.248868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:55.258632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:55.260030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:55.269796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1326/2659] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:55.318377 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:55.319734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:55.331964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20802c-6aec-4b13-91df-e4cc8296694a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:55.335530 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:55.336123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:55.336401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:55.336670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:55.376733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:55.395291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:55.396718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edd419-8eea-4da0-9bc9-0c670edfc72d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:55.405347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1334/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:58.763370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:58.804306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:58.826309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c25c-6d84-455e-8167-262b534049a1 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:58.831512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:58.832109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:58.832375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:58.832639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] PATH_INFO: `/v3/users/3a99824bba794269828769fb3ac0b32a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:58.833794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a99824bba794269828769fb3ac0b32a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:58.835935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:58.879687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a99824bba794269828769fb3ac0b32a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:58.880324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a99824bba794269828769fb3ac0b32a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:58.880911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a99824bba794269828769fb3ac0b32a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:58.887701 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a99824bba794269828769fb3ac0b32a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:58.900073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a99824bba794269828769fb3ac0b32a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:58.913289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1327/2661] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:58 2022] DELETE /identity/v3/users/3a99824bba794269828769fb3ac0b32a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:58.917766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:58.918189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:58.918315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:58.937620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:32:58.938805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:58.996973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1335/2662] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:32:59.006910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:59.008322 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:59.066755 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11dfe-005f-4e3f-9742-14411cfd6a83 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:59.070395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:59.070994 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:59.071272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:59.071542 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:59.080466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:59.082756 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:59.092132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1328/2663] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:32:59.364110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:59.365610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:32:59.377143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5eb8-f5ec-40e4-9633-e122079a2454 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:59.381793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:59.383027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:59.383027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:59.383027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] PATH_INFO: `/v3/projects/7a7ca27719b64c19ac8aaed0c1872b36` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:59.384085 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7ca27719b64c19ac8aaed0c1872b36)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:59.386017 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:59.434096 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7ca27719b64c19ac8aaed0c1872b36', 'request_id': 'req-361e9025-e016-4891-b120-d6cca673e7b4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.440106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7ca27719b64c19ac8aaed0c1872b36', 'request_id': 'req-361e9025-e016-4891-b120-d6cca673e7b4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.447940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7ca27719b64c19ac8aaed0c1872b36', 'request_id': 'req-361e9025-e016-4891-b120-d6cca673e7b4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.459665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] The token cache is being invalidate because project 7a7ca27719b64c19ac8aaed0c1872b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:32:59.459962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.460603 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.463557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1336/2664] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:59 2022] DELETE /identity/v3/projects/7a7ca27719b64c19ac8aaed0c1872b36 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:32:59.467498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:59.477937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73f06-5e22-4a18-aff4-e0869bbc714d service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:32:59.535111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:32:59.535750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:32:59.536019 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:32:59.536287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] PATH_INFO: `/v3/users/7b914e02dad7459daeb991dfd13a8765` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:32:59.537564 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b914e02dad7459daeb991dfd13a8765)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:32:59.539632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:32:59.541243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:32:59.595740 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b914e02dad7459daeb991dfd13a8765'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.596459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b914e02dad7459daeb991dfd13a8765'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.597798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b914e02dad7459daeb991dfd13a8765'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.605533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b914e02dad7459daeb991dfd13a8765'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.621742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-541d46f2-1a74-44aa-b34a-1b020fc69362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b914e02dad7459daeb991dfd13a8765'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:32:59.641002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1329/2665] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:32:59 2022] DELETE /identity/v3/users/7b914e02dad7459daeb991dfd13a8765 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:32:59.993285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:32:59.994911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:00.008840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e9025-e016-4891-b120-d6cca673e7b4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.074237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.074881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.075161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.075438 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] PATH_INFO: `/v3/projects/aa3fe97a862f4be08f173ed799b81e54` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.076748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3fe97a862f4be08f173ed799b81e54)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.078780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.086174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:00.134187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3fe97a862f4be08f173ed799b81e54', 'request_id': 'req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:00.140022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3fe97a862f4be08f173ed799b81e54', 'request_id': 'req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:00.148536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3fe97a862f4be08f173ed799b81e54', 'request_id': 'req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:00.160385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] The token cache is being invalidate because project aa3fe97a862f4be08f173ed799b81e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:33:00.160666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:00.161301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cdba7f50-82cb-4e22-bc2d-2fa44a1eb781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:00.162747 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1337/2666] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:32:59 2022] DELETE /identity/v3/projects/aa3fe97a862f4be08f173ed799b81e54 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:00.167540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.168056 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.168150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.188562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:00.189783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:00.249434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1330/2667] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:00.257616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.257900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.258177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.278714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:00.279917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:00.288691 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1338/2668] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:00.292897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.303169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f89fc-2c49-4149-9508-0e004918a6ff None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.308017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.308616 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.308889 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.309188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.309825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:33:00.310123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.311687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.327573 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1331/2669] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:33:00.328526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.338648 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc64fd-2fe9-43fa-ae8d-b313f01a41f3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.343348 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.343940 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.344211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.344477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.345223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.347904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.376138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1339/2670] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:00.380614 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.390382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436e3-66f5-4eca-9709-a956e7fb0a23 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.451075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.451694 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.451997 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.452286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.452971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.454878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.467510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:00.483192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1332/2671] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:00.484552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.496146 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f34dc-c5de-43f9-94e5-8cd72af27eba admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.500708 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.501331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.501602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.501892 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.502522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.504168 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.514094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1340/2672] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:00.518161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.531142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5c334-436d-4069-8a5f-202eae8fc861 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.537796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.538790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.539066 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.539337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.539980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.541633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97c6fa-fe12-434a-a24d-6b136c1aae31 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.552836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1333/2673] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:33:00.557060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.568385 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99969fb9-b031-4db5-b1be-1eb1e9942259 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.574254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.575251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.575535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.575810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] PATH_INFO: `/v3/projects/2dc7c47ff832466cb6bf031ca85e2161/users/3e5e7aebf3784d98bf20cfa260c1dbd1/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.593857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc7c47ff832466cb6bf031ca85e2161, user_id=3e5e7aebf3784d98bf20cfa260c1dbd1, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.596517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.611515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1341/2674] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:00 2022] PUT /identity/v3/projects/2dc7c47ff832466cb6bf031ca85e2161/users/3e5e7aebf3784d98bf20cfa260c1dbd1/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:00.622672 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.623086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.623434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.651643 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] MFA Rules not processed for user `3e5e7aebf3784d98bf20cfa260c1dbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:00.652834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:00.716394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1334/2675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:00.720841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.782046 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded9259-f876-413c-bc0a-936d3efca8e5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.787251 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.787851 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.788124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.788388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.789040 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.790917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.818783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1342/2676] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:00.822798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.834682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5acf2-7def-4447-ac98-1d923a69dc68 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.896390 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.897029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.897313 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.897601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.898222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.900123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.923096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1335/2677] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:33:00 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:00.927873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.938450 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41b3a-8ca6-4a7f-8bc6-ad56bf2cfca9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.943083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.943670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.943946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.944215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:00.944845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:00.946469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:00.957023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1343/2678] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:00.961131 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:00.971043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1c2e5-948c-418f-8b03-93c64d4d1dff admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:00.976915 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:00.977584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:00.977859 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:00.978178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] PATH_INFO: `/v3/projects/d5e2832071a44e13938ac1301464322e/users/a328a76a31b44759aa99f776c24b82dc/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:01.000126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e2832071a44e13938ac1301464322e, user_id=a328a76a31b44759aa99f776c24b82dc, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:01.002833 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:01.015586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1336/2679] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:00 2022] PUT /identity/v3/projects/d5e2832071a44e13938ac1301464322e/users/a328a76a31b44759aa99f776c24b82dc/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:33:01.020147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:01.032587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:01.037968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18a45-960b-4efa-bb0a-d716dd3bef6f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:01.096083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:01.097174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:01.097460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:01.097757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:01.098388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:01.099969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d7c06-05a2-4eba-add5-73e793900353 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:01.109395 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1344/2680] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:01.113736 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:01.123946 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f0a2c-a7c3-4a19-a403-294a979798b5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:01.128738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:01.129404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:01.129679 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:01.129948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] PATH_INFO: `/v3/projects/d5e2832071a44e13938ac1301464322e/users/a328a76a31b44759aa99f776c24b82dc/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:01.131854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e2832071a44e13938ac1301464322e, user_id=a328a76a31b44759aa99f776c24b82dc, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:01.134487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:01.145108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1337/2681] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:01 2022] PUT /identity/v3/projects/d5e2832071a44e13938ac1301464322e/users/a328a76a31b44759aa99f776c24b82dc/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:33:01.155084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:01.155371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:01.155644 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:01.183488 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] MFA Rules not processed for user `a328a76a31b44759aa99f776c24b82dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:01.184684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:01.253383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1345/2682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:01.264050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:01.265851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:01.297844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:01.325037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65dfe7-b2e0-4c28-b952-96576792c418 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:01.328603 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:01.329223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:01.329498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:01.329765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:01.384186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:01.385575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a47c1-75b7-4153-bc9e-4af1f08ad22d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:01.394944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1338/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:33:03.966946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:04.059277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64e54-c879-462b-b92e-651fa467419d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:04.064452 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:04.066174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:04.066552 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:04.066946 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] PATH_INFO: `/v3/users/3e5e7aebf3784d98bf20cfa260c1dbd1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:04.068216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5e7aebf3784d98bf20cfa260c1dbd1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:04.070402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:04.103266 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:04.125974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5e7aebf3784d98bf20cfa260c1dbd1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.126776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5e7aebf3784d98bf20cfa260c1dbd1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.134600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5e7aebf3784d98bf20cfa260c1dbd1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.135441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5e7aebf3784d98bf20cfa260c1dbd1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.148677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5e7aebf3784d98bf20cfa260c1dbd1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.162504 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1346/2684] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:33:03 2022] DELETE /identity/v3/users/3e5e7aebf3784d98bf20cfa260c1dbd1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:04.167159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:04.167728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:04.167728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:04.189331 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:04.190663 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:04.252734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1339/2685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:04.262925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:04.264388 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:04.336461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a139-d28a-4c6f-b697-88a3e1614029 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:04.340526 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:04.341269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:04.341683 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:04.342032 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:04.352684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:04.358577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:04.371339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1347/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:33:04.730483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:04.731945 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:04.745340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889a60-3dfc-4e4f-9c1f-a221d400db50 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:04.751150 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:04.753548 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:04.757470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:04.757470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] PATH_INFO: `/v3/projects/2dc7c47ff832466cb6bf031ca85e2161` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:04.758857 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc7c47ff832466cb6bf031ca85e2161)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:04.761074 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:04.825854 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc7c47ff832466cb6bf031ca85e2161', 'request_id': 'req-e6b0de27-9524-442f-b668-065554978d03'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.832111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc7c47ff832466cb6bf031ca85e2161', 'request_id': 'req-e6b0de27-9524-442f-b668-065554978d03'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.842149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc7c47ff832466cb6bf031ca85e2161', 'request_id': 'req-e6b0de27-9524-442f-b668-065554978d03'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.853866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] The token cache is being invalidate because project 2dc7c47ff832466cb6bf031ca85e2161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:33:04.854179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.854814 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:04.856309 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1340/2687] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:33:04 2022] DELETE /identity/v3/projects/2dc7c47ff832466cb6bf031ca85e2161 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:04.860230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:04.871729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849fb23-8eb3-4fe3-ba4d-f3846fa5ea1b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:04.936414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:04.937338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:04.937620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:04.937891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] PATH_INFO: `/v3/users/a328a76a31b44759aa99f776c24b82dc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:04.942097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a328a76a31b44759aa99f776c24b82dc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:04.944134 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:05.000952 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a328a76a31b44759aa99f776c24b82dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.002102 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a328a76a31b44759aa99f776c24b82dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.004920 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:05.011829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a328a76a31b44759aa99f776c24b82dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.012682 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a328a76a31b44759aa99f776c24b82dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.031801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4f17101c-dcbc-4cca-98e0-c95842fea311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a328a76a31b44759aa99f776c24b82dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.052872 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1348/2688] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:33:04 2022] DELETE /identity/v3/users/a328a76a31b44759aa99f776c24b82dc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:05.366416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:05.368539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:05.385440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:05.407987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0de27-9524-442f-b668-065554978d03 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:05.496623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:05.497568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.497742 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.498008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] PATH_INFO: `/v3/projects/d5e2832071a44e13938ac1301464322e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.499143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e2832071a44e13938ac1301464322e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:05.501059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:05.570601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e2832071a44e13938ac1301464322e', 'request_id': 'req-63c3e040-2106-4e01-82f4-3c3b132785da'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.579929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e2832071a44e13938ac1301464322e', 'request_id': 'req-63c3e040-2106-4e01-82f4-3c3b132785da'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.592667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e2832071a44e13938ac1301464322e', 'request_id': 'req-63c3e040-2106-4e01-82f4-3c3b132785da'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.614477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] The token cache is being invalidate because project d5e2832071a44e13938ac1301464322e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:33:05.615136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.616073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-63c3e040-2106-4e01-82f4-3c3b132785da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:05.621797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1341/2689] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:33:05 2022] DELETE /identity/v3/projects/d5e2832071a44e13938ac1301464322e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:05.625427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.625711 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.625982 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.653094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:05.655902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:05.745497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1349/2690] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:05.755372 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.755714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.756033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.779093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:05.780610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:05.789822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1342/2691] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:05.794080 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:05.806578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7917e3-57d3-4fd2-9fe1-c26d94062d01 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:05.811343 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:05.812341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.812719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.813189 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.813961 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:33:05.814360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:05.816721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:05.833592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1350/2692] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:33:05.834847 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:05.848771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3829-e3cd-4fad-8da0-789631b87521 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:05.855098 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:05.856599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.856885 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.857907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.859046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:05.861137 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:05.897078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1343/2693] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:33:05 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:05.901073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:05.913334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84e9a1-8384-4599-8d22-9f99cb00b875 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:05.979115 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:05.979883 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:05.980260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:05.980632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:05.981413 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:05.983483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:06.016039 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1351/2694] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:33:05 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:33:06.020324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:06.046119 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d7f62-cedc-4eb6-bfe7-09d66dab29f6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:06.054169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:06.054845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.055138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.055396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.056048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:06.057768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:06.074151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1344/2695] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:33:06.075729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:06.089663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4b499-eba2-43f0-967c-570c5fd59366 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:06.095370 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:06.096122 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.097063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.097289 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.097942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:06.099973 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467ee62-1fa4-4a23-b075-a1f286a16797 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:06.113555 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1352/2696] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:06.117373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:06.130176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3393430-e60b-4ff8-a0c8-5c4753954f17 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:06.135499 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:06.136222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.136626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.136918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] PATH_INFO: `/v3/projects/f5a7235827c243c9828ebca72e47c156/users/691641e056634995888c88395d0fb47f/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.156912 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a7235827c243c9828ebca72e47c156, user_id=691641e056634995888c88395d0fb47f, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:06.160211 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:06.182826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1345/2697] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:06 2022] PUT /identity/v3/projects/f5a7235827c243c9828ebca72e47c156/users/691641e056634995888c88395d0fb47f/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:06.252887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2cbc-08e0-458c-b247-a490f0f091e2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.253268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2cbc-08e0-458c-b247-a490f0f091e2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.253546 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2cbc-08e0-458c-b247-a490f0f091e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.287225 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-172f2cbc-08e0-458c-b247-a490f0f091e2 None None] MFA Rules not processed for user `691641e056634995888c88395d0fb47f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:06.288436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-172f2cbc-08e0-458c-b247-a490f0f091e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:06.309607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:06.363287 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1353/2698] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:06.374142 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:06.376053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:06.418691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:06.448579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b467d-f1ee-4712-8106-97327d010f37 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:06.452891 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:06.453533 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.453813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.454087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.463591 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:06.465072 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be070f-feb0-4187-86b0-f4c517b00a62 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:06.475454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1346/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:33:06.598580 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663d41-a052-4804-b731-9a54cfdf1945 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.599685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663d41-a052-4804-b731-9a54cfdf1945 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.600174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663d41-a052-4804-b731-9a54cfdf1945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.622881 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f3663d41-a052-4804-b731-9a54cfdf1945 None None] MFA Rules not processed for user `a5433685dfeb41abaf7c58758390ad9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:06.624578 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3663d41-a052-4804-b731-9a54cfdf1945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:06.765015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1354/2700] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:06.781563 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447978c-6f76-41c3-ac59-b7fcb39ed068 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:06.781794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447978c-6f76-41c3-ac59-b7fcb39ed068 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:06.782070 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447978c-6f76-41c3-ac59-b7fcb39ed068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:06.850584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0447978c-6f76-41c3-ac59-b7fcb39ed068 None None] MFA Rules not processed for user `04cf365443b44d9d87338d95b5b0251b`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:06.851869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0447978c-6f76-41c3-ac59-b7fcb39ed068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:06.993091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1347/2701] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:33:08.070581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3fc1-969e-4fe2-8519-171f7775979d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:08.071706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3fc1-969e-4fe2-8519-171f7775979d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:08.072199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3fc1-969e-4fe2-8519-171f7775979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:08.104237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-300b3fc1-969e-4fe2-8519-171f7775979d None None] MFA Rules not processed for user `691641e056634995888c88395d0fb47f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:08.106171 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-300b3fc1-969e-4fe2-8519-171f7775979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:08.119737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1355/2702] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:16.978934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=37 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:18.095837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=45 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:20.090430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:20.090909 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:20.091001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:20.111079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] MFA Rules not processed for user `44adfe9a391948f68dc747448546063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:20.112794 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:20.170354 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1348/2703] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:30.163350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=25 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:31.973411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032513f-9667-4394-8ca4-1780b4379f3f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:31.973958 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032513f-9667-4394-8ca4-1780b4379f3f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:31.974090 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032513f-9667-4394-8ca4-1780b4379f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:31.994051 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e032513f-9667-4394-8ca4-1780b4379f3f None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:31.995337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e032513f-9667-4394-8ca4-1780b4379f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:32.053896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1356/2704] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:33:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:32.102433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:32.104017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:32.178334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee4eb6-7f1a-4ef1-9b2c-616a8cb0d6fa None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:32.183477 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:32.184086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:32.184365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:32.184637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:32.193431 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:32.195039 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:32.203094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1349/2705] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:33:32.764684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289e-843d-42bb-819a-e38a997bc900 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:32.765057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289e-843d-42bb-819a-e38a997bc900 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:32.765393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289e-843d-42bb-819a-e38a997bc900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:32.784839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a70f289e-843d-42bb-819a-e38a997bc900 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:32.786180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a70f289e-843d-42bb-819a-e38a997bc900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:32.793820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1357/2706] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:33:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:32.806724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:32.818071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e31e0-b0d8-411f-8fd6-ac06c98eb109 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:32.822998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:32.823602 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:32.823874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:32.824143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:32.833092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:32.834682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:32.843625 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1350/2707] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:33:33.083804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb0078-c8ad-4f33-8ada-6ae4d469cfec None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:33.084311 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb0078-c8ad-4f33-8ada-6ae4d469cfec None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:33.084501 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb0078-c8ad-4f33-8ada-6ae4d469cfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:33.104245 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1bbb0078-c8ad-4f33-8ada-6ae4d469cfec None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:33.105562 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1bbb0078-c8ad-4f33-8ada-6ae4d469cfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:33.113626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1358/2708] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:33:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:33.125555 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:33.136528 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22837c-ff63-4ace-92ae-559af50d1ad8 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:33.141524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:33.142155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:33.142434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:33.142729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:33.151194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:33.152766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:33.161770 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1351/2709] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:33:33.505018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:33.505372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:33.505714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:33.526272 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:33.527487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:33.535882 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1359/2710] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:33:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:33.548041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:33.560037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08c32-7ee8-4099-b06b-10d57330885d service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:33.566105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:33.566718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:33.566999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:33.567274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:33.576316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:33.577901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053d0bc-febc-4274-825e-19f94a858fc0 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:33.587705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1352/2711] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:33:35.750155 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:35.755411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:35.755869 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:35.756079 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:35.779305 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:35.780615 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:35.788856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1353/2712] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 07:33:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:33:35.802214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:35.815508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e876935-5047-49d6-9793-80cfc427bd68 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:35.820101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93a59d-fcb8-4965-b5c0-a0cf1d15d303 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:35.820836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:35.821470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:35.821754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:35.822028 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:35.824558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:35.825214 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:35.825491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:35.825758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] PATH_INFO: `/v3/users/691641e056634995888c88395d0fb47f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:35.826957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] RBAC: Authorizing `identity:delete_user(user_id=691641e056634995888c88395d0fb47f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:35.828852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:35.831731 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:35.833377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21853725-1db0-43ad-af66-5db4b8e6d105 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:35.841670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1354/2713] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 07:33:35.882268 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691641e056634995888c88395d0fb47f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:35.883195 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691641e056634995888c88395d0fb47f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:35.891309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691641e056634995888c88395d0fb47f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:35.892188 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691641e056634995888c88395d0fb47f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:35.907582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691641e056634995888c88395d0fb47f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:35.925221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1360/2714] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:33:35 2022] DELETE /identity/v3/users/691641e056634995888c88395d0fb47f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:35.929707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:35.930254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:35.930777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:35.952162 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:35.953493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:36.031802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1355/2715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:36.042362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.044611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:36.070485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:36.162136 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f493f4-3808-4762-af1e-969c2060873e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.184260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.185346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.185780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.186232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.199684 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.201671 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e25ce3-cce1-4fb6-8ff7-0d689c7302ce service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.214532 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1361/2716] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:33:36.454970 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.456887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:36.471524 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bef79-02dc-43d3-ba77-f35cecf48574 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.478391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.479040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.479316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.479587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] PATH_INFO: `/v3/projects/f5a7235827c243c9828ebca72e47c156` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.482084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a7235827c243c9828ebca72e47c156)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.483971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.526949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:36.547517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a7235827c243c9828ebca72e47c156', 'request_id': 'req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:36.553782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a7235827c243c9828ebca72e47c156', 'request_id': 'req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:36.562154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a7235827c243c9828ebca72e47c156', 'request_id': 'req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:36.575626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] The token cache is being invalidate because project f5a7235827c243c9828ebca72e47c156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:33:36.575999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:36.576657 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cdeb6f4e-5c13-49a4-b64c-23438ba6e619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:33:36.578451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1356/2717] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:33:36 2022] DELETE /identity/v3/projects/f5a7235827c243c9828ebca72e47c156 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:36.583968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.584529 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.585106 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.606320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:36.607715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:36.675417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1362/2718] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:36.682393 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.682671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.682941 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.713135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:36.714355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:36.723080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1357/2719] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:33:36.727342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.738455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2223-2097-4ac4-a6af-d09b4d6c45a7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.743624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.744990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.744990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.744990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.745409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:33:36.745710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.747316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.764358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1363/2720] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:33:36.768802 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.778479 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d318c-988b-4048-90d3-a85a216d5ef3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.785877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.786709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.787059 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.787459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.788115 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.790089 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.824116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1358/2721] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:33:36 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:36.827599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.839809 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67de24-0741-41e4-822b-c8d23bc3d3ee admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.912534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.912534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.912534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.912534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.913107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.915232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.941932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1364/2722] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:33:36 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:36.946096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.960411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fc906-f119-4b9c-95ae-57692845ee1f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:36.966276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:36.967383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:36.967825 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:36.968096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:36.968738 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:36.970386 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:36.987160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:36.992566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1359/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:36.997679 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9b1cf-0c1a-459b-88da-f1f842fb860f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:37.005397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:37.006000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:37.006267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:37.006969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:37.007635 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:37.009524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707f9f-b2f1-42fb-961f-4266ccc75487 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:37.021506 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1365/2724] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:33:37.025682 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:37.036205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fedd1-e5c4-4035-8792-7f094040fc99 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:37.041163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:37.041820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:37.042105 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:37.042453 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] PATH_INFO: `/v3/projects/39ec4a1f0d60440c86413595a8eb16ae/users/880d5318ebb145fbb3cd8b02814903f2/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:37.059271 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ec4a1f0d60440c86413595a8eb16ae, user_id=880d5318ebb145fbb3cd8b02814903f2, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:37.064035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:37.089009 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1360/2725] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:37 2022] PUT /identity/v3/projects/39ec4a1f0d60440c86413595a8eb16ae/users/880d5318ebb145fbb3cd8b02814903f2/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:33:37.092338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620974-2018-4921-b57e-a0d211e0b1d3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:37.092629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620974-2018-4921-b57e-a0d211e0b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:37.092900 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620974-2018-4921-b57e-a0d211e0b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:37.119224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b2620974-2018-4921-b57e-a0d211e0b1d3 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:37.120573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2620974-2018-4921-b57e-a0d211e0b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:37.149472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:37.239525 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1366/2726] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:37.249881 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:37.251633 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:37.327474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54687-bb67-4329-a512-bf0790aeb33c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:37.332368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:37.333378 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:37.333786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:37.334194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:37.347185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:37.348610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:37.361841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1361/2727] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:33:42.712697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:42.713548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:42.713818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:42.756901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] MFA Rules not processed for user `880d5318ebb145fbb3cd8b02814903f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:42.758197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:42.854638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1367/2728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:42.856165 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:42.868513 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d8d34-4251-4bdf-8a11-77cf4840022b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:42.891193 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:42.891804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:42.892077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:42.892349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:42.893189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:42.895244 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:42.929439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1362/2729] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:33:42 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:42.933317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:42.953989 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b2cd9-39e5-4376-981c-87722ee1ba85 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:43.053309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:43.054092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:43.054688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:43.055113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:43.055865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:43.058244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:43.098696 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1368/2730] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:33:42 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:33:43.100155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:43.119974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488fa3-5be9-42c1-aad4-11302ac26d61 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:43.127531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:43.128265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:43.128728 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:43.129060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:43.129795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:43.131535 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:43.149171 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1363/2731] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:43.151765 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:43.170237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921915a8-1503-4ff6-885d-db47dfdccd47 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:43.177496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:43.178632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:43.179030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:43.180019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] PATH_INFO: `/v3/projects/365db8c3711b4a718745dfd83a505234/users/b5d4df6ebec743b991eac4eee5cad9bf/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:43.201627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=365db8c3711b4a718745dfd83a505234, user_id=b5d4df6ebec743b991eac4eee5cad9bf, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:43.242321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:43.269021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1369/2732] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:43 2022] PUT /identity/v3/projects/365db8c3711b4a718745dfd83a505234/users/b5d4df6ebec743b991eac4eee5cad9bf/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:33:43.273574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:43.304836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf600-b2da-4401-9548-e186ec59a8c8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:43.335793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:43.450370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:43.451408 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:43.451910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:43.452384 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:43.456099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:43.458646 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab13f7d-f9b0-4e8f-9d25-dc949c5a4efb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:43.491490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1364/2733] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:33:43.492043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:43.510036 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d355c-4a93-4454-92e4-fbe1a0155e6c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:43.544328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:43.544328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:43.544717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:43.544837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] PATH_INFO: `/v3/projects/365db8c3711b4a718745dfd83a505234/users/b5d4df6ebec743b991eac4eee5cad9bf/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:43.550277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] RBAC: Authorizing `identity:create_grant(project_id=365db8c3711b4a718745dfd83a505234, user_id=b5d4df6ebec743b991eac4eee5cad9bf, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:43.553884 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:43.558996 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:33:43.571336 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1370/2734] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:33:43 2022] PUT /identity/v3/projects/365db8c3711b4a718745dfd83a505234/users/b5d4df6ebec743b991eac4eee5cad9bf/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:33:50.181036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab135c7-ff46-4f1b-a65f-c10222f5770a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:50.181545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab135c7-ff46-4f1b-a65f-c10222f5770a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:50.181545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab135c7-ff46-4f1b-a65f-c10222f5770a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:50.217055 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ab135c7-ff46-4f1b-a65f-c10222f5770a None None] MFA Rules not processed for user `b5d4df6ebec743b991eac4eee5cad9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:50.218374 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ab135c7-ff46-4f1b-a65f-c10222f5770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:50.471623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1365/2735] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:33:50.488441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:33:50.490234 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:50.594217 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b407f-6e5b-41ea-b596-934cfcf3832d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:33:50.602240 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:33:50.603375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:50.603663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:50.603939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:50.676980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:33:50.678780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46dac20-6f1e-4e5e-bcad-5ea30f238ff7 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:33:50.703917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1371/2736] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:33:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:33:52.255381 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:33:52.256018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:33:52.256018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:33:52.282688 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] MFA Rules not processed for user `880d5318ebb145fbb3cd8b02814903f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:33:52.284012 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:33:52.291765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1366/2737] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:33:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:34:00.684536 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=58 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:01.641257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:01.641541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:01.641810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:01.661309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] MFA Rules not processed for user `a3ccebac846245ee915636757eb921f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:01.662482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:01.724470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1372/2738] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 07:34:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:34:01.736554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:01.738623 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:01.805416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eba403-9b32-4a3a-8665-48abe2361029 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:01.810346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4389bd42da64dccbf958503dccf87e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:01.810938 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:01.811212 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:01.811481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:01.819774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:01.821433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:01.829186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1367/2739] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 07:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 07:34:03.756924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:03.758490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:03.841955 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aa8e0-3855-4ea5-befc-6ae163492fd4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:03.846417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:03.847039 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:03.847314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:03.848126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] PATH_INFO: `/v3/users/a5433685dfeb41abaf7c58758390ad9b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:03.849346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5433685dfeb41abaf7c58758390ad9b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:03.851241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:03.896162 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5433685dfeb41abaf7c58758390ad9b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:03.896812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5433685dfeb41abaf7c58758390ad9b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:03.903804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5433685dfeb41abaf7c58758390ad9b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:03.904430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5433685dfeb41abaf7c58758390ad9b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:03.916737 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2f813e6-4733-4957-9451-fe7b9ab7ac93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5433685dfeb41abaf7c58758390ad9b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:03.929440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1373/2740] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:34:03 2022] DELETE /identity/v3/users/a5433685dfeb41abaf7c58758390ad9b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:34:04.101812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.112766 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8202dd-dc66-4789-b1cf-33a8574e2dac service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.136543 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:04.175297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.176092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.176490 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.176764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] PATH_INFO: `/v3/projects/01b7cf3a259443a9a370b79e2bc43dcc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.177905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b7cf3a259443a9a370b79e2bc43dcc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.179793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.224878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b7cf3a259443a9a370b79e2bc43dcc', 'request_id': 'req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:04.230291 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b7cf3a259443a9a370b79e2bc43dcc', 'request_id': 'req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:04.237540 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b7cf3a259443a9a370b79e2bc43dcc', 'request_id': 'req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:04.248512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] The token cache is being invalidate because project 01b7cf3a259443a9a370b79e2bc43dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:34:04.248793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:04.249433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2e744c47-fc6f-44f3-b871-6c6b6d6bfb79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:34:04.250864 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1368/2741] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:34:04 2022] DELETE /identity/v3/projects/01b7cf3a259443a9a370b79e2bc43dcc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:34:04.255925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88196-2981-4e85-8033-b8211641b1df None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.256286 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88196-2981-4e85-8033-b8211641b1df None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.256571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88196-2981-4e85-8033-b8211641b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.279730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fac88196-2981-4e85-8033-b8211641b1df None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:04.281111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fac88196-2981-4e85-8033-b8211641b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:04.301734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:04.342785 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1374/2742] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:34:04.351323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.351606 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.351879 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.370975 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:04.372173 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:04.380357 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1369/2743] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:34:04.384486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac88196-2981-4e85-8033-b8211641b1df None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.393748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac88196-2981-4e85-8033-b8211641b1df None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.398853 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.399444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.399716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.399983 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.400616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:34:04.400916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.402520 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.413799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1375/2744] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:34:04.417999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.427174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c64998-44a8-4d31-b5f5-10565055bdce None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.432169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.432754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.433061 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.433326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.433948 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.435812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.471692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1370/2745] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:34:04 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:34:04.476244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.487279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c3451-af0a-499e-9074-0dd6208a5858 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.547064 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.547746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.548094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.548418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.549157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.551340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.573484 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1376/2746] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:34:04 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:34:04.577583 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.588935 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01531f5-1c57-4635-a3af-d1f0417d4342 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.593483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.594128 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.594437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.594698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.595323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.596921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.606562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1371/2747] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:34:04.610342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.621309 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5298b-42f6-4f97-b60c-ca80e16cfe82 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.625824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.626409 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.626681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.626947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.627577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.629714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533406a7-cb98-46fd-8058-f83deebe717e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.640041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1377/2748] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:34:04.644122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.654455 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127c7a-2810-49c3-9e10-ff961b20883b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.659011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.659637 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.659990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.660407 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] PATH_INFO: `/v3/projects/8e8bbfbde29f4df1ba0d4ea32e8a15d4/users/d7ff2b03e1d940d28cf8e7f1c675ff09/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.674906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8bbfbde29f4df1ba0d4ea32e8a15d4, user_id=d7ff2b03e1d940d28cf8e7f1c675ff09, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.678161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.696640 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1372/2749] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:34:04 2022] PUT /identity/v3/projects/8e8bbfbde29f4df1ba0d4ea32e8a15d4/users/d7ff2b03e1d940d28cf8e7f1c675ff09/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:34:04.701686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08306a-a40c-47e2-9d63-599b4554bf1d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.702057 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08306a-a40c-47e2-9d63-599b4554bf1d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.702364 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08306a-a40c-47e2-9d63-599b4554bf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.725411 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6f08306a-a40c-47e2-9d63-599b4554bf1d None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:04.726642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6f08306a-a40c-47e2-9d63-599b4554bf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:04.785330 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1378/2750] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:34:04.795047 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:04.796754 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:04.845696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:04.858866 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3020c-a143-4a13-90d1-17956273d92d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:04.862621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:04.863774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:04.864080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:04.864411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:04.873270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:04.874687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:04.885200 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1373/2751] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:05.797301 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b87a-fa53-4565-93b9-21e10b4a1795 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:05.797834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b87a-fa53-4565-93b9-21e10b4a1795 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:05.797947 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b87a-fa53-4565-93b9-21e10b4a1795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:05.826666 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a188b87a-fa53-4565-93b9-21e10b4a1795 None None] MFA Rules not processed for user `d7ff2b03e1d940d28cf8e7f1c675ff09`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:05.827962 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a188b87a-fa53-4565-93b9-21e10b4a1795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:05.847376 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:05.903300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1379/2752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:34:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:34:05.914387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:05.915748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:05.977253 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece728-2e54-4fa2-bffe-455550c8cecd service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:05.980823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:05.981451 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:05.981762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:05.982030 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:05.990498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:05.991848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:06.001262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1374/2753] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:07.374979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:07.375527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:07.375527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:07.394868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] MFA Rules not processed for user `d7ff2b03e1d940d28cf8e7f1c675ff09`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:34:07.396043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:07.402857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1380/2754] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:34:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:34:07.445905 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:07.447328 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:07.502685 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403247fb-46b2-4272-a62c-fbf1996b8aec service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:07.506246 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca1a72134341ed9dbe8b58c339873e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:07.506831 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:07.507101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:07.507367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:07.509058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:07.576680 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:07.578041 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:07.588782 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1375/2755] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:08.116339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:08.126475 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77fdfc-30ce-46cc-badb-7bb71861c95d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:08.130432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:08.131029 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:08.131292 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:08.131729 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:08.133858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:08.150042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:08.151524 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:08.163514 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1381/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:17.583914 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=87 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:18.160026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=27 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:31.652628 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:31.661761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5895-1996-49b3-880e-699bce6cae6c service glance] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:31.665278 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:31.665858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:31.666127 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:31.666388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:31.668036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:31.729986 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:31.731405 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb68c0-6bb3-4abe-b351-60f1616b5746 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:31.741496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1376/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:33.667685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:34:33.676105 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe730602-62c5-4132-8aa2-94c5872fe5dd service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:34:33.680565 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:34:33.681199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:34:33.681474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:34:33.681744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:34:33.683479 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:34:33.694885 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:34:33.696533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:34:33.706312 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1382/2758] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:34:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:34:41.737111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=30 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:34:43.702571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:03.872767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031313d-13fc-444c-9a97-dacf1394f5b4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:03.873298 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031313d-13fc-444c-9a97-dacf1394f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:03.873417 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031313d-13fc-444c-9a97-dacf1394f5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:03.892811 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6031313d-13fc-444c-9a97-dacf1394f5b4 None None] MFA Rules not processed for user `44adfe9a391948f68dc747448546063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:35:03.894040 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6031313d-13fc-444c-9a97-dacf1394f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:03.951054 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1377/2759] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:35:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:35:09.389787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:09.392419 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:09.411387 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d27d5-4d9f-4ef2-80e8-f3f857babad3 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:09.417625 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:09.418469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:09.418749 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:09.419020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:09.420796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:09.436782 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:09.438730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:09.451389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1383/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:35:13.944111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=25 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:19.445596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:19.505871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:19.506434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:19.506434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:19.535248 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] MFA Rules not processed for user `d7ff2b03e1d940d28cf8e7f1c675ff09`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:35:19.536521 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:19.543686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1378/2761] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:35:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:35:19.793992 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:19.795426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:19.807247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa1c5-eda7-4984-b3a0-b6098c67f7b2 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:19.810811 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:19.811397 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:19.811681 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:19.812204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:19.813904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:19.823486 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:19.824956 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:19.833534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1384/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:35:20.142817 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:20.144231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:20.215023 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e039d48-3dec-4adf-9986-313bd8aabf81 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:20.219971 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:20.220571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:20.220845 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:20.221151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:20.223642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:20.235574 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:20.237295 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:20.248155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1379/2763] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:35:29.573793 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:29.588663 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ffc7b-1bf8-4c4a-8c81-2831c5e2df85 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:29.594424 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:29.595021 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:29.595297 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:29.595566 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:29.597945 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:29.610190 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:29.611757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76b0c2-7131-478f-8f5d-d33be1c9b058 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:29.624328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1385/2764] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:35:30.243834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=35 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:38.737092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:38.738604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:38.750662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc8e1b-46b4-4d3c-b35e-cae20cc8901c service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:38.754778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:38.755761 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:38.756035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:38.756302 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:38.758093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:38.818213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:38.819624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:38.828699 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1380/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:35:39.618676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=12 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:48.824338 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=28 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:35:55.152399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:55.152934 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:55.153070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:55.172344 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] MFA Rules not processed for user `04cf365443b44d9d87338d95b5b0251b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:35:55.173719 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:55.181615 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1386/2766] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:35:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:35:55.497655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:35:55.506818 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73d9c-2c8a-4e69-98c5-6ef5eedea28a service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:35:55.510310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:35:55.510900 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:35:55.511169 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:35:55.511433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:35:55.513334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:35:55.522609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:35:55.524004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:35:55.532043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1381/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:35:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:05.167263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:05.528225 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:18.808238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:18.819126 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1bcc-5830-468c-97de-a8c700cc3122 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:18.822626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:18.823212 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:18.823485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:18.823750 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:18.825456 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:18.887279 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:18.888697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:18.898478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1387/2768] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:20.991890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:20.993523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:21.011786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873c80-4017-4fb8-a6d8-d50266ed4da0 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:21.018299 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:21.018906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:21.019186 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:21.019509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:21.021807 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:21.034135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:21.036392 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:21.047507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1382/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:27.832444 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:27.843659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7afaa-8d1b-43bc-9faa-e0dd0b6403a4 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:27.849043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:27.849652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:27.849924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:27.850191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:27.851897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:27.863874 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:27.865480 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:27.876367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1388/2770] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:27.889512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:27.900644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb6f91-937e-40a0-b599-14043ede8f3d service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:27.906619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:27.907352 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:27.907512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:27.908090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] PATH_INFO: `/v3/users/d7ff2b03e1d940d28cf8e7f1c675ff09` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:27.909109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ff2b03e1d940d28cf8e7f1c675ff09)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:27.911353 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:27.958950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ff2b03e1d940d28cf8e7f1c675ff09'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:27.959653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ff2b03e1d940d28cf8e7f1c675ff09'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:27.960252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ff2b03e1d940d28cf8e7f1c675ff09'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:27.967097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ff2b03e1d940d28cf8e7f1c675ff09'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:27.978834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ff2b03e1d940d28cf8e7f1c675ff09'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:27.992509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1383/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:27 2022] DELETE /identity/v3/users/d7ff2b03e1d940d28cf8e7f1c675ff09 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:28.161688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:28.174787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a554-501d-4807-9d60-aaeb006f1743 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:28.232909 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:28.233558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:28.233831 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:28.234094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] PATH_INFO: `/v3/projects/8e8bbfbde29f4df1ba0d4ea32e8a15d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:28.235180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8bbfbde29f4df1ba0d4ea32e8a15d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:28.237082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:28.281756 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8bbfbde29f4df1ba0d4ea32e8a15d4', 'request_id': 'req-f1c6a59a-0636-49ba-b573-778bb51c26fe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:28.288295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8bbfbde29f4df1ba0d4ea32e8a15d4', 'request_id': 'req-f1c6a59a-0636-49ba-b573-778bb51c26fe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:28.295642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8bbfbde29f4df1ba0d4ea32e8a15d4', 'request_id': 'req-f1c6a59a-0636-49ba-b573-778bb51c26fe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:28.307533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] The token cache is being invalidate because project 8e8bbfbde29f4df1ba0d4ea32e8a15d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:28.307812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:28.308440 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:28.309938 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1389/2772] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:28 2022] DELETE /identity/v3/projects/8e8bbfbde29f4df1ba0d4ea32e8a15d4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:37.987993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:38.304439 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:40.780432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:40.781893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:40.839827 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b909f11-9963-4a17-86e0-0d0cf41e2049 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:40.889460 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:40.890053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:40.890324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:40.890594 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:40.892259 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:40.955190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:40.956514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:40.966464 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1384/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:44.093454 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.095023 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:44.109953 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a59a-0636-49ba-b573-778bb51c26fe admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.114846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.115472 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.115766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.116020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] PATH_INFO: `/v3/users/44adfe9a391948f68dc747448546063e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.117202 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=44adfe9a391948f68dc747448546063e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.119160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.166020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44adfe9a391948f68dc747448546063e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.166911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44adfe9a391948f68dc747448546063e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.175588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44adfe9a391948f68dc747448546063e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.176455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44adfe9a391948f68dc747448546063e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.193243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ff860285-798a-46e0-8111-ef8b840ff33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44adfe9a391948f68dc747448546063e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.206236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1390/2774] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:44 2022] DELETE /identity/v3/users/44adfe9a391948f68dc747448546063e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:44.378834 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.390440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8cf01-693a-457d-ad61-75db369962b3 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.478404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.478957 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.479228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.479493 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] PATH_INFO: `/v3/projects/94a0b09b62ce47678fd542a0453c3d17` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.480670 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a0b09b62ce47678fd542a0453c3d17)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.487415 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.495664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:44.532961 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a0b09b62ce47678fd542a0453c3d17', 'request_id': 'req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.539257 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a0b09b62ce47678fd542a0453c3d17', 'request_id': 'req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.546433 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a0b09b62ce47678fd542a0453c3d17', 'request_id': 'req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.557930 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] The token cache is being invalidate because project 94a0b09b62ce47678fd542a0453c3d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:44.558238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.559015 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-73d96175-b4d0-44f6-b69b-2886d0f2e5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:44.560820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1385/2775] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:44 2022] DELETE /identity/v3/projects/94a0b09b62ce47678fd542a0453c3d17 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:44.566027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.566317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.566614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.586703 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:44.587901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:44.645422 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1391/2776] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:44.653243 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.653545 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.653812 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.674303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:44.675505 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:44.683420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1386/2777] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:44.687847 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.698236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8b32f-247f-49f8-a690-300f68cf93dd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.702868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.703470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.703748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.704020 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.704661 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:36:44.704984 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.707180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.718842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1392/2778] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:36:44.723226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.733906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e20852-0855-47b6-928b-1b3a3b7fedba None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.738559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.739147 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.739418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.739706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.740333 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.742213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.770443 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1387/2779] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:44 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:36:44.774672 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.789221 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464a2da-39a6-402f-9389-1ef6d3bd4660 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.847991 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.848616 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.848888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.849184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.849817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.851761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.859559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:44.879230 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1393/2780] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:36:44 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:44.880636 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.890931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd780b1-17b3-48d5-bfa2-53b0e097e384 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.896335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.896913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.897215 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.897486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.898120 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.899696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.909749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1388/2781] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:44.914086 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.924054 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565067-78ef-46c1-97b7-d3b0d38da41b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.928767 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.929417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.929677 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.929951 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.930587 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.932198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443501e-8bba-46ec-9204-89b0929f125a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.943596 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1394/2782] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:44.947512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:44.958203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0855b89-d61a-477f-8925-36d8806e48dc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:44.962691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:44.963324 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:44.963595 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:44.963990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] PATH_INFO: `/v3/projects/fdabdd286f0b4aa083228d83ab2da629/users/7e1a7b864768445dbce9b2d2a6212fa1/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:44.977871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdabdd286f0b4aa083228d83ab2da629, user_id=7e1a7b864768445dbce9b2d2a6212fa1, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:44.980510 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:44.992897 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1389/2783] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:44 2022] PUT /identity/v3/projects/fdabdd286f0b4aa083228d83ab2da629/users/7e1a7b864768445dbce9b2d2a6212fa1/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:45.002932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:45.003243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:45.004193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:45.031768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] MFA Rules not processed for user `7e1a7b864768445dbce9b2d2a6212fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:45.032988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:45.098810 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1395/2784] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:36:45.111116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:45.113498 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:45.192871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598272-ff6c-4d35-9d21-6a4504f15ea4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:45.198502 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e403816a5674d45b3cefdc34cffcf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:45.199104 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:45.199383 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:45.199653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:45.207411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:45.209033 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177ff6-1029-4531-b539-1d0db65b9a67 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:45.217660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1390/2785] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:53.535367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:53.595142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9799-3038-4256-b922-0e93445e9167 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:53.599582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:53.600193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:53.600467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:53.600734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] PATH_INFO: `/v3/users/7e1a7b864768445dbce9b2d2a6212fa1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:53.601855 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1a7b864768445dbce9b2d2a6212fa1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:53.603734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:53.648789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1a7b864768445dbce9b2d2a6212fa1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:53.649458 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1a7b864768445dbce9b2d2a6212fa1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:53.656243 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1a7b864768445dbce9b2d2a6212fa1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:53.656865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1a7b864768445dbce9b2d2a6212fa1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:53.667276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:53.669724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1a7b864768445dbce9b2d2a6212fa1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:53.684199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1396/2786] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:53 2022] DELETE /identity/v3/users/7e1a7b864768445dbce9b2d2a6212fa1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:36:53.689092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:53.689567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:53.689767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:53.709013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:53.710196 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:53.721695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:53.769223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1391/2787] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:36:53.780018 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:53.781461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:53.839942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1bb6-dbba-47c1-8e91-369e2d237d0e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:53.843559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:53.844156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:53.844433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:53.844697 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:53.854112 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:53.855462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc249e8-5d85-4259-b618-a5ac1b689afb service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:53.865461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1397/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:54.139167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.140561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:54.153402 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15582c4-fd53-4326-8cb2-32d4d981f557 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.158060 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.158687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.158965 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.159223 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] PATH_INFO: `/v3/projects/fdabdd286f0b4aa083228d83ab2da629` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.160343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdabdd286f0b4aa083228d83ab2da629)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.162820 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.211902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdabdd286f0b4aa083228d83ab2da629', 'request_id': 'req-a4151caa-9b52-4f87-8cf3-a2dcaad24645'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:54.217784 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdabdd286f0b4aa083228d83ab2da629', 'request_id': 'req-a4151caa-9b52-4f87-8cf3-a2dcaad24645'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:54.225691 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdabdd286f0b4aa083228d83ab2da629', 'request_id': 'req-a4151caa-9b52-4f87-8cf3-a2dcaad24645'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:54.236832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] The token cache is being invalidate because project fdabdd286f0b4aa083228d83ab2da629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:54.237160 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:54.237797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a4151caa-9b52-4f87-8cf3-a2dcaad24645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:54.239724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1392/2789] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:54 2022] DELETE /identity/v3/projects/fdabdd286f0b4aa083228d83ab2da629 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:54.244516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.244799 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.245098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.265620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:54.266812 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:54.325483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1398/2790] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:54.335360 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.335644 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.335920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.356233 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:54.357448 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:54.365647 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1393/2791] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:54.370030 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.380260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce204-ca1e-4029-ac9e-04929b687c04 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.385179 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.385715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.385990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.386254 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.386887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:36:54.387183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.388783 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.404321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1399/2792] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:36:54.405425 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.414715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978169-72d9-401e-a486-cde6692c5eb0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.419665 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.420268 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.420546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.420816 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.421481 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.423365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.452421 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1394/2793] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:36:54.456698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.468143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4801fa-f4ab-4ce4-8bd3-e9b77852d1a3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.527948 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.528629 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.529003 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.529340 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.530070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.531964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.551846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1400/2794] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:36:54.557559 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.570418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b110-a383-4195-83f9-302c4d807abd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.574974 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.575565 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.575836 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.576101 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.576735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.578368 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.588760 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1395/2795] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:54.592763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.602801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80c82-d680-4f90-be68-fa272187b3c6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.607307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.607888 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.608157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.608434 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.609087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.610652 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4158ce-5c33-4ab7-892f-88bf7483dcdf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.611818 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:54.623215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1401/2796] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:54.627236 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.637853 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584dbc-ba8a-4145-8a92-a644db03b37b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.642325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.642956 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.643227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.643491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] PATH_INFO: `/v3/projects/adcadebc653141aaae09b68614897ba3/users/1b560b8e462e4d3eaa191d80cdaac0c0/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.658925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=adcadebc653141aaae09b68614897ba3, user_id=1b560b8e462e4d3eaa191d80cdaac0c0, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.661917 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.673921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1396/2797] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:54 2022] PUT /identity/v3/projects/adcadebc653141aaae09b68614897ba3/users/1b560b8e462e4d3eaa191d80cdaac0c0/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:54.684241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.684521 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.685084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.715789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] MFA Rules not processed for user `1b560b8e462e4d3eaa191d80cdaac0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:54.717304 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:54.780964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1402/2798] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:54.785497 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.826733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:54.849151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87624b-5b4c-4a29-9342-23e8d31014ba admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.853734 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.854321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.854593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.854858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.855484 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.857370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.884262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1397/2799] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:54.888356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.898822 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130726b0-abdb-4665-81d2-cd4a0e833d02 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.953858 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.954447 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.954721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.954987 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.955610 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:54.957589 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:54.978037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1403/2800] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:36:54 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:54.981989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:54.992806 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a5ca-9c86-4ffb-9471-5f77617c6050 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:54.997394 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:54.997979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:54.998251 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:54.998515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:54.999141 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.000689 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.011424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1398/2801] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:55.015359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.025477 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154be169-f2d3-40bd-8a1b-dc2ef39ac691 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.030084 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.030727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.031006 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.031280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] PATH_INFO: `/v3/projects/7586a394d7fb4f26abae1450ed66a4c8/users/677d977e68cd49bc88bb21b0a78f0bd9/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.045808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7586a394d7fb4f26abae1450ed66a4c8, user_id=677d977e68cd49bc88bb21b0a78f0bd9, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.048550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.062430 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1404/2802] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:55 2022] PUT /identity/v3/projects/7586a394d7fb4f26abae1450ed66a4c8/users/677d977e68cd49bc88bb21b0a78f0bd9/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:55.066485 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.076933 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e959822-b4c3-4449-94bc-a04756f672f6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.131791 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.132387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.132662 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.132929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.133593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.135192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485faec-7ac4-4096-88f7-282a17ccfb24 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.144671 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1399/2803] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:55.148826 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.159232 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85849f2e-795e-48ac-9f41-08666a7f375f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.164571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.165256 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.165537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.165804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] PATH_INFO: `/v3/projects/7586a394d7fb4f26abae1450ed66a4c8/users/677d977e68cd49bc88bb21b0a78f0bd9/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.167715 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7586a394d7fb4f26abae1450ed66a4c8, user_id=677d977e68cd49bc88bb21b0a78f0bd9, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.170359 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.182517 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1405/2804] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:55 2022] PUT /identity/v3/projects/7586a394d7fb4f26abae1450ed66a4c8/users/677d977e68cd49bc88bb21b0a78f0bd9/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:55.192918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.193238 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.193515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.220630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] MFA Rules not processed for user `677d977e68cd49bc88bb21b0a78f0bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:55.221993 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:55.288052 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1400/2805] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:55.300276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.302631 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:55.344590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:55.360879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3087e-a130-45b2-bcbd-dce642f9df70 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.365246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.365840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.366117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.366384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.375246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.376674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.385972 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1406/2806] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:55.700939 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.710651 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4f62d-b711-440d-9dc6-a358ab685eb4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.715326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677d977e68cd49bc88bb21b0a78f0bd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7586a394d7fb4f26abae1450ed66a4c8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.715918 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.716203 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.716478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.717163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:36:55.717478 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.719046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.730549 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1401/2807] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:36:55.734723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.745156 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd684989-b8e4-4fbc-97ba-d1ff75c00441 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.749839 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677d977e68cd49bc88bb21b0a78f0bd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7586a394d7fb4f26abae1450ed66a4c8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.750428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.750707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.750980 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.751617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.753568 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.780414 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1407/2808] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:55 2022] POST /identity/v3/projects => generated 352 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:36:55.784496 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.795176 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580236-51db-436c-9f0d-6409cae2ac01 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.812227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:55.853650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677d977e68cd49bc88bb21b0a78f0bd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7586a394d7fb4f26abae1450ed66a4c8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.854241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.854518 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.854788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.855416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:36:55.855715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.857317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.868209 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1402/2809] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:36:55.904969 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:55.915159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b8680-4f1c-4c1d-a1b7-f54261a6b345 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:55.919744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677d977e68cd49bc88bb21b0a78f0bd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7586a394d7fb4f26abae1450ed66a4c8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:55.920296 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:55.920573 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:55.920845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] PATH_INFO: `/v3/projects/816254d9de85472ba709b337c143077b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:55.922016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorizing `identity:delete_project(project_id=816254d9de85472ba709b337c143077b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:55.923880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:55.972187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816254d9de85472ba709b337c143077b', 'request_id': 'req-cee3e8cf-f804-4d54-9fc5-0340382bc363'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:55.978864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816254d9de85472ba709b337c143077b', 'request_id': 'req-cee3e8cf-f804-4d54-9fc5-0340382bc363'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:55.985821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816254d9de85472ba709b337c143077b', 'request_id': 'req-cee3e8cf-f804-4d54-9fc5-0340382bc363'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:55.997203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] The token cache is being invalidate because project 816254d9de85472ba709b337c143077b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:55.997485 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:55.998119 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cee3e8cf-f804-4d54-9fc5-0340382bc363 tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:55.999591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1408/2810] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:55 2022] DELETE /identity/v3/projects/816254d9de85472ba709b337c143077b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:56.200355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:56.202566 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:56.266940 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f816b2-1eea-4db7-be61-44b2e32ff17f tempest-VolumeQuotaClassesTest-839122818 tempest-VolumeQuotaClassesTest-839122818-project-admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:56.321887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:56.322568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:56.322841 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:56.323109 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] PATH_INFO: `/v3/users/1b560b8e462e4d3eaa191d80cdaac0c0` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:56.324254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b560b8e462e4d3eaa191d80cdaac0c0)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:56.326174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:56.370174 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b560b8e462e4d3eaa191d80cdaac0c0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.370823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b560b8e462e4d3eaa191d80cdaac0c0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.371428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b560b8e462e4d3eaa191d80cdaac0c0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.378187 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b560b8e462e4d3eaa191d80cdaac0c0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.384049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:56.392184 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b560b8e462e4d3eaa191d80cdaac0c0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.404580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1403/2811] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:56 2022] DELETE /identity/v3/users/1b560b8e462e4d3eaa191d80cdaac0c0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:56.409060 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:56.409346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:56.409647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:56.430094 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:56.431241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:56.493128 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1409/2812] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:56.502861 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:56.504275 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:56.561495 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf4b5a-4122-46c8-bbcf-5894ac10c66b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:56.565146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:56.565733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:56.566002 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:56.566265 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:56.575709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:56.577140 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:56.586349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1404/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:56.873642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:56.875093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:56.890854 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056af150-b27f-433d-b49f-bbc2865c88bb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:56.899662 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:56.900326 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:56.900602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:56.900869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] PATH_INFO: `/v3/projects/adcadebc653141aaae09b68614897ba3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:56.902062 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] RBAC: Authorizing `identity:delete_project(project_id=adcadebc653141aaae09b68614897ba3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:56.903902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:56.942421 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:56.952727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adcadebc653141aaae09b68614897ba3', 'request_id': 'req-da806596-c681-437b-8f72-b4327fb3132d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.958761 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adcadebc653141aaae09b68614897ba3', 'request_id': 'req-da806596-c681-437b-8f72-b4327fb3132d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.965687 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adcadebc653141aaae09b68614897ba3', 'request_id': 'req-da806596-c681-437b-8f72-b4327fb3132d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.977469 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] The token cache is being invalidate because project adcadebc653141aaae09b68614897ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:56.977752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.978372 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:56.979828 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1410/2814] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:56 2022] DELETE /identity/v3/projects/adcadebc653141aaae09b68614897ba3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:56.983904 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:56.994049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78aeda0-04bb-4f92-802d-aa46da34738a service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.048509 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.049155 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.049430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.049698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] PATH_INFO: `/v3/users/677d977e68cd49bc88bb21b0a78f0bd9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.050795 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677d977e68cd49bc88bb21b0a78f0bd9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.052796 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.100143 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677d977e68cd49bc88bb21b0a78f0bd9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.100781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677d977e68cd49bc88bb21b0a78f0bd9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.101410 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677d977e68cd49bc88bb21b0a78f0bd9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.108590 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677d977e68cd49bc88bb21b0a78f0bd9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.121069 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcda3695-f293-43a9-888a-e1332afff8a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677d977e68cd49bc88bb21b0a78f0bd9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.133725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1405/2815] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:56 2022] DELETE /identity/v3/users/677d977e68cd49bc88bb21b0a78f0bd9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:57.401685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.403211 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:57.416768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806596-c681-437b-8f72-b4327fb3132d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.477360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.478026 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.478302 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.478571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] PATH_INFO: `/v3/projects/7586a394d7fb4f26abae1450ed66a4c8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.479689 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7586a394d7fb4f26abae1450ed66a4c8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.481705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.527380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7586a394d7fb4f26abae1450ed66a4c8', 'request_id': 'req-a4588a7f-3324-4923-85c2-4321ee62137d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.532846 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7586a394d7fb4f26abae1450ed66a4c8', 'request_id': 'req-a4588a7f-3324-4923-85c2-4321ee62137d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.539432 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7586a394d7fb4f26abae1450ed66a4c8', 'request_id': 'req-a4588a7f-3324-4923-85c2-4321ee62137d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.550445 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] The token cache is being invalidate because project 7586a394d7fb4f26abae1450ed66a4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:36:57.550725 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.551338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a4588a7f-3324-4923-85c2-4321ee62137d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:57.552837 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1411/2816] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:57 2022] DELETE /identity/v3/projects/7586a394d7fb4f26abae1450ed66a4c8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:57.557893 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.558180 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.558474 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.566310 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:57.580871 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:57.582095 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:57.641222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1406/2817] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:57.705191 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.705474 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.705744 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.725590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:57.726771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:57.734508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1412/2818] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:57.738546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.749125 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28837b2-3558-49b4-8c39-9b7807b45d6b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.753562 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.754151 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.754422 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.754721 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.755501 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:36:57.755798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.757430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.768660 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1407/2819] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:36:57.773647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.782748 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b77c2-2047-4543-825b-4f606e576f61 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.788333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.788925 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.789238 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.789511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.790143 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.792149 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.820158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1413/2820] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:57 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:36:57.824277 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.834861 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058b64-7da5-4387-86cb-5c0ac252232f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.889195 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.889789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.890063 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.890332 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.890959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.893008 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.916667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1408/2821] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:36:57 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:57.920771 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.930933 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe259e7-ad8f-478f-940d-e8dd4c0cbe7a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.935784 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.936380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.936650 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.936917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.937613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.939184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.953158 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1414/2822] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:36:57.954032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.965025 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712f121-318b-4302-ae74-b2f34132c8fb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:57.969598 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:57.970188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:57.970458 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:57.970723 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:57.971470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:57.973065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e1591-3142-4c8a-bdcf-1a36963e7fbf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:57.984504 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1409/2823] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:57.988624 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:57.999448 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b04fd-1b4a-494f-8579-5b1a82d8c0a2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.004053 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.004762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.005070 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.005342 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] PATH_INFO: `/v3/projects/a3d6588a4f5b408a872c0f56289d9e36/users/f66a2cac100145cd95297e132b61439d/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.017600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:58.022015 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d6588a4f5b408a872c0f56289d9e36, user_id=f66a2cac100145cd95297e132b61439d, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.025544 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.039825 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1415/2824] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:57 2022] PUT /identity/v3/projects/a3d6588a4f5b408a872c0f56289d9e36/users/f66a2cac100145cd95297e132b61439d/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:58.049877 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.050231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.050587 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.079469 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] MFA Rules not processed for user `f66a2cac100145cd95297e132b61439d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:58.080664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:58.139091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:58.146339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1410/2825] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:58.150668 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.211901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da9fff-29aa-4fad-9291-7f19b40cbb82 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.216433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.217068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.217347 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.217614 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.218233 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.220110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.246196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1416/2826] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:36:58 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:58.250414 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.261375 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7fa7-bf9b-4078-a4d5-46debfc382f8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.316232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.316911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.317724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.317998 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.318624 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.320515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.341423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1411/2827] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:36:58 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:36:58.345541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.356059 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf5ba-0a55-46d5-9cac-6eb68f192e5d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.360647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.361285 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.361567 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.361841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.362482 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.364095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.374515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1417/2828] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:58.378430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.388931 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33ef0-ffa6-42a9-9727-14ed2671428f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.393487 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.394116 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.394389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.394656 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] PATH_INFO: `/v3/projects/d5e688fb907747ae8953ef5546fcd382/users/782fa731f56046cba4a3b834097104b3/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.409396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e688fb907747ae8953ef5546fcd382, user_id=782fa731f56046cba4a3b834097104b3, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.412250 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.424436 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1412/2829] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:58 2022] PUT /identity/v3/projects/d5e688fb907747ae8953ef5546fcd382/users/782fa731f56046cba4a3b834097104b3/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:58.428418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.438534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce128a-5125-4694-a96f-a515f39cf12d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.495617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.496264 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.496487 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.496758 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.497428 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.499140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96e09-554f-4ff2-be36-98f4a7ddb29a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.509263 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1418/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:36:58.513454 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.523878 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1489b78-e1d6-4a98-af6a-19f94e965e0a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.528855 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.529531 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.529804 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.530090 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] PATH_INFO: `/v3/projects/d5e688fb907747ae8953ef5546fcd382/users/782fa731f56046cba4a3b834097104b3/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.531999 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e688fb907747ae8953ef5546fcd382, user_id=782fa731f56046cba4a3b834097104b3, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.534767 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.545430 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1413/2831] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:36:58 2022] PUT /identity/v3/projects/d5e688fb907747ae8953ef5546fcd382/users/782fa731f56046cba4a3b834097104b3/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:58.555606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.555887 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.556159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.584393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] MFA Rules not processed for user `782fa731f56046cba4a3b834097104b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:58.585611 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:58.632718 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:58.654463 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1419/2832] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:36:58.666619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:58.668396 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:58.726588 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009ac8-3215-4760-86a5-e5f2041bae2d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:58.730297 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:58.730901 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:58.731181 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:58.731456 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:58.741035 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:58.742465 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001f347-631b-464e-a5e5-b6d3f7c9ad4d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:58.752161 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1414/2833] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:36:59.329224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:59.388824 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bafa7-0fd7-47a1-996c-f9a574b5d06f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:59.393299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:59.393918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:59.394196 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:59.394467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] PATH_INFO: `/v3/users/f66a2cac100145cd95297e132b61439d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:59.395863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66a2cac100145cd95297e132b61439d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:59.397796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:59.444633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66a2cac100145cd95297e132b61439d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:59.445470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66a2cac100145cd95297e132b61439d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:59.452310 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66a2cac100145cd95297e132b61439d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:59.452937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66a2cac100145cd95297e132b61439d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:59.464834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66a2cac100145cd95297e132b61439d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:36:59.477271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1420/2834] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:36:59 2022] DELETE /identity/v3/users/f66a2cac100145cd95297e132b61439d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:36:59.481798 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:59.482287 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:59.482391 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:59.501210 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:36:59.502423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:59.548447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:36:59.562122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1415/2835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:36:59.574142 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:59.575545 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:59.631082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade83a-e9d9-4ac8-9fba-47cd5a40ebe0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:59.634658 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:59.635253 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:59.635522 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:59.635786 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:59.644507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:59.646147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:59.655273 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1421/2836] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:36:59.918750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:36:59.920149 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:36:59.934411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca6631-ddfe-408c-84c1-f45ea0e8d377 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:36:59.940316 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:36:59.940973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:36:59.941252 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:36:59.941541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] PATH_INFO: `/v3/projects/a3d6588a4f5b408a872c0f56289d9e36` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:36:59.943743 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d6588a4f5b408a872c0f56289d9e36)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:36:59.945653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:36:59.995774 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d6588a4f5b408a872c0f56289d9e36', 'request_id': 'req-3c063925-167f-47f9-8d13-0893b346eb7c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.001385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d6588a4f5b408a872c0f56289d9e36', 'request_id': 'req-3c063925-167f-47f9-8d13-0893b346eb7c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.009444 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d6588a4f5b408a872c0f56289d9e36', 'request_id': 'req-3c063925-167f-47f9-8d13-0893b346eb7c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.021486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] The token cache is being invalidate because project a3d6588a4f5b408a872c0f56289d9e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:00.021732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.022370 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.024087 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1416/2837] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:36:59 2022] DELETE /identity/v3/projects/a3d6588a4f5b408a872c0f56289d9e36 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:37:00.027879 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.037341 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d179c-6df0-483d-a1ef-64db1b84e690 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.041979 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:00.095260 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.095805 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.096083 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.096352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] PATH_INFO: `/v3/users/782fa731f56046cba4a3b834097104b3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.097483 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782fa731f56046cba4a3b834097104b3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.099766 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.149510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782fa731f56046cba4a3b834097104b3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.150160 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782fa731f56046cba4a3b834097104b3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.157295 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782fa731f56046cba4a3b834097104b3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.157924 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782fa731f56046cba4a3b834097104b3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.169690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c4d0bb7-889f-4f94-bb04-e29376105ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782fa731f56046cba4a3b834097104b3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.183038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1422/2838] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:00 2022] DELETE /identity/v3/users/782fa731f56046cba4a3b834097104b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:00.465991 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.467674 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:00.486168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c063925-167f-47f9-8d13-0893b346eb7c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.547228 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.547849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.548123 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.548388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] PATH_INFO: `/v3/projects/d5e688fb907747ae8953ef5546fcd382` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.549579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e688fb907747ae8953ef5546fcd382)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.552082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.598325 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e688fb907747ae8953ef5546fcd382', 'request_id': 'req-f246b565-286c-4f3a-857e-ca99c05aa8b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.603645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e688fb907747ae8953ef5546fcd382', 'request_id': 'req-f246b565-286c-4f3a-857e-ca99c05aa8b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.610936 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e688fb907747ae8953ef5546fcd382', 'request_id': 'req-f246b565-286c-4f3a-857e-ca99c05aa8b0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.622412 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] The token cache is being invalidate because project d5e688fb907747ae8953ef5546fcd382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:00.622697 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.623318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f246b565-286c-4f3a-857e-ca99c05aa8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:00.624984 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1417/2839] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:00 2022] DELETE /identity/v3/projects/d5e688fb907747ae8953ef5546fcd382 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:00.630476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.630977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.631293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.651558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:00.652753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:00.712140 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1423/2840] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:00.719658 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.720032 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.720308 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.734432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:00.741017 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:00.742801 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:00.751097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1418/2841] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:00.755863 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.765541 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89b0b0-a1f0-4237-8039-1d3bf47e63d9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.770117 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.771351 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.771632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.771896 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.772535 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:37:00.772838 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.774489 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.786250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1424/2842] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:37:00.790783 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.800065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e17cb7-9b19-415f-a8b2-ce0fda1ab27f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.805584 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.806179 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.806457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.806730 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.807397 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.809343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.838289 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1419/2843] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:00 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:00.842473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.852491 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3e25-59a4-4f02-9f28-6c713f8e7416 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.863441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:00.909174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.909762 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.910037 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.910305 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.910921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.912820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.934427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1425/2844] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:00 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:37:00.938896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.950247 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53f79-f144-458b-a671-e9792d524ddd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.954823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.955411 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.955692 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.955963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.956604 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.958725 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:00.968417 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1420/2845] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:00.972367 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:00.982241 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2e564-5371-4ff3-8bcc-ff57aa32cd8c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:00.986717 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:00.987299 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:00.987599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:00.987868 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:00.988495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:00.990620 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfad7e-10f4-43bd-a11b-413471fc1744 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.001093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1426/2846] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:01.005747 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.015619 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5fd6c-e59c-433e-b429-0c40f2fb0121 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.020260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.021073 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.021350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.021621 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] PATH_INFO: `/v3/projects/64440b4b1551429fbd441f81bf5fa795/users/20d13ecb4e754aadb496da0026826f87/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.036213 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64440b4b1551429fbd441f81bf5fa795, user_id=20d13ecb4e754aadb496da0026826f87, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.038848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.051894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1421/2847] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:01 2022] PUT /identity/v3/projects/64440b4b1551429fbd441f81bf5fa795/users/20d13ecb4e754aadb496da0026826f87/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:37:01.062306 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.062593 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.062865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.090667 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] MFA Rules not processed for user `20d13ecb4e754aadb496da0026826f87`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:01.091864 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:01.154690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1427/2848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:01.159084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.225355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef3c8c-e74e-4a90-ad3e-7f353d2d05a1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.229963 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.230550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.230823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.231088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.231719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.233617 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.259764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1422/2849] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:01 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:01.263817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.275113 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a52b-d3f8-4b01-a555-31e0ef763362 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.332172 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.333441 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.333712 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.334007 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.334636 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.336626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.357800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1428/2850] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:01 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:37:01.361848 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.372003 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc811-2518-454a-9a14-cda5ea54ae1c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.376593 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.377205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.377483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.377752 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.378389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.380466 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.390608 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1423/2851] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:37:01.394678 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.405317 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345af9b-4748-4e96-b112-7230feb40679 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.409804 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.410462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.410724 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.411000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] PATH_INFO: `/v3/projects/25018955f72f4077ae38b0f749eb31f6/users/3ef59c1afc9f4a75b4d03c04311f2ff2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.421901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:01.426779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25018955f72f4077ae38b0f749eb31f6, user_id=3ef59c1afc9f4a75b4d03c04311f2ff2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.430223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.445220 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1429/2852] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:01 2022] PUT /identity/v3/projects/25018955f72f4077ae38b0f749eb31f6/users/3ef59c1afc9f4a75b4d03c04311f2ff2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:01.449157 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.459388 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c5884-998f-4298-b470-366e66faf6bb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.515753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.516459 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.516649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.517092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.517874 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.519475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f273f9-cf80-4df0-b413-0c0c4d61286b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.530508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1424/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:01.534857 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.545010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bc6ec-a7b0-4236-9e3b-ca5d7579c12e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.549557 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.550198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.550471 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.550738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] PATH_INFO: `/v3/projects/25018955f72f4077ae38b0f749eb31f6/users/3ef59c1afc9f4a75b4d03c04311f2ff2/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.553321 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25018955f72f4077ae38b0f749eb31f6, user_id=3ef59c1afc9f4a75b4d03c04311f2ff2, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.555911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.566974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1430/2854] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:01 2022] PUT /identity/v3/projects/25018955f72f4077ae38b0f749eb31f6/users/3ef59c1afc9f4a75b4d03c04311f2ff2/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:01.578580 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.578973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.579317 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.605321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:01.609053 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] MFA Rules not processed for user `3ef59c1afc9f4a75b4d03c04311f2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:01.611000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:01.680349 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1425/2855] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:01.691204 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:01.693676 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:01.749727 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5900d-5f15-4a1e-aeeb-1504439211e0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:01.753361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:01.753954 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:01.754229 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:01.755111 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:01.763904 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:01.765357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0fe08-e263-4c7c-a702-22e3e5815016 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:01.774794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1431/2856] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:02.149506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:02.221153 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd142c24-45b1-4f8e-8803-13e983fbea70 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:02.226130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:02.226764 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:02.227036 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:02.227303 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] PATH_INFO: `/v3/users/20d13ecb4e754aadb496da0026826f87` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:02.228428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d13ecb4e754aadb496da0026826f87)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:02.230483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:02.279194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d13ecb4e754aadb496da0026826f87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.279846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d13ecb4e754aadb496da0026826f87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.280434 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d13ecb4e754aadb496da0026826f87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.287526 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d13ecb4e754aadb496da0026826f87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.299314 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d13ecb4e754aadb496da0026826f87'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.313949 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1426/2857] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:02 2022] DELETE /identity/v3/users/20d13ecb4e754aadb496da0026826f87 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:02.319145 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:02.319507 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:02.319728 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:02.338829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:02.340041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:02.399184 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1432/2858] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:02.409117 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:02.410515 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:02.430678 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:02.471082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d70032-7445-4dce-bca4-bab0b1228a05 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:02.476483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:02.477058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:02.477340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:02.477612 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:02.487065 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:02.488541 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:02.498944 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1427/2859] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:02.829242 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:02.830739 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:02.842591 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f403-9dac-4366-8150-ab92c878cbe5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:02.847222 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:02.847840 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:02.848693 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:02.849186 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] PATH_INFO: `/v3/projects/64440b4b1551429fbd441f81bf5fa795` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:02.850460 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64440b4b1551429fbd441f81bf5fa795)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:02.852228 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:02.899334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64440b4b1551429fbd441f81bf5fa795', 'request_id': 'req-707d6123-c96d-4b8b-8452-dec51fa30ed0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.904813 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64440b4b1551429fbd441f81bf5fa795', 'request_id': 'req-707d6123-c96d-4b8b-8452-dec51fa30ed0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.912561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64440b4b1551429fbd441f81bf5fa795', 'request_id': 'req-707d6123-c96d-4b8b-8452-dec51fa30ed0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.923772 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] The token cache is being invalidate because project 64440b4b1551429fbd441f81bf5fa795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:02.924058 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.924685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:02.926199 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1433/2860] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:02 2022] DELETE /identity/v3/projects/64440b4b1551429fbd441f81bf5fa795 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:02.930664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:02.940473 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897069b0-d45c-44f8-bde3-8582814aaef7 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:02.996227 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:02.996768 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:02.997071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:02.997342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] PATH_INFO: `/v3/users/3ef59c1afc9f4a75b4d03c04311f2ff2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:02.998470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef59c1afc9f4a75b4d03c04311f2ff2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.000358 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.046000 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef59c1afc9f4a75b4d03c04311f2ff2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.046635 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef59c1afc9f4a75b4d03c04311f2ff2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.047226 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef59c1afc9f4a75b4d03c04311f2ff2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.055138 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef59c1afc9f4a75b4d03c04311f2ff2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.067748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-baae8461-fc58-43b5-927f-e23e126c0dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef59c1afc9f4a75b4d03c04311f2ff2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.080529 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1428/2861] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:02 2022] DELETE /identity/v3/users/3ef59c1afc9f4a75b4d03c04311f2ff2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:03.341216 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.342673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:03.350627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:03.358307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6123-c96d-4b8b-8452-dec51fa30ed0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.422848 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.423470 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.423746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.424022 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] PATH_INFO: `/v3/projects/25018955f72f4077ae38b0f749eb31f6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.425147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25018955f72f4077ae38b0f749eb31f6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.428082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.473598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25018955f72f4077ae38b0f749eb31f6', 'request_id': 'req-474f08d4-9ab1-4935-9f5f-779372918e41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.479135 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25018955f72f4077ae38b0f749eb31f6', 'request_id': 'req-474f08d4-9ab1-4935-9f5f-779372918e41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.486239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25018955f72f4077ae38b0f749eb31f6', 'request_id': 'req-474f08d4-9ab1-4935-9f5f-779372918e41'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.498320 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] The token cache is being invalidate because project 25018955f72f4077ae38b0f749eb31f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:03.498607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.499236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-474f08d4-9ab1-4935-9f5f-779372918e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:03.500702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1434/2862] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:03 2022] DELETE /identity/v3/projects/25018955f72f4077ae38b0f749eb31f6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:03.506014 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.506367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.506645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.526097 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:03.527234 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:03.560858 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:03.588178 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1429/2863] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:03.595577 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.595861 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.596130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.615490 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:03.616675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:03.624511 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1435/2864] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:03.628762 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.638972 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87b8fe-a64a-494f-8764-8da6c454378f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.643511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.644094 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.644366 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.644632 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.645934 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:37:03.646231 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.647826 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.658921 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1430/2865] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:37:03.663789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.672746 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11af40-9b04-4d96-8e47-55c350b75a51 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.677870 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.678455 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.678726 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.678990 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.679617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.681538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.710466 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1436/2866] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:03 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:03.714599 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.725340 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5955c3-0236-4c30-a8bd-83ce84ba1eee admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.780744 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.781437 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.781715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.781980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.782645 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.784537 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.806735 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1431/2867] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:03 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:37:03.811103 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.821723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871cdcd-83f1-465c-baf8-2cbfbd6fd5e1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.826582 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.827177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.827453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.827722 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.828361 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.829978 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.840378 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1437/2868] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:03.844427 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.855470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d839a-9ec3-4ce9-bae0-29981e16dadc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.860083 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.860669 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.860964 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.861239 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.861926 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.863519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e807e-7587-4e0d-bfb7-7d4ecb2e6d56 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.874214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1432/2869] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:37:03.878237 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:03.888527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0ba9b-7ff2-4306-806e-ddde72957796 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:03.893389 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:03.894078 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.894358 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.894633 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] PATH_INFO: `/v3/projects/88f1e4d66efc498992919fa2540fadb3/users/e5b243e3ae794120ba1a8b3161a17eac/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.910400 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f1e4d66efc498992919fa2540fadb3, user_id=e5b243e3ae794120ba1a8b3161a17eac, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:03.914043 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:03.928012 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1438/2870] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:03 2022] PUT /identity/v3/projects/88f1e4d66efc498992919fa2540fadb3/users/e5b243e3ae794120ba1a8b3161a17eac/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:03.933192 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a3d8-f5d4-4c38-812f-ae19a63b24c6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:03.933745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a3d8-f5d4-4c38-812f-ae19a63b24c6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:03.933745 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a3d8-f5d4-4c38-812f-ae19a63b24c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:03.953718 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-52e9a3d8-f5d4-4c38-812f-ae19a63b24c6 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:03.954929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-52e9a3d8-f5d4-4c38-812f-ae19a63b24c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:04.016695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1433/2871] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:04.026927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:04.028602 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:04.063803 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:04.090716 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204302c-41eb-4f48-9b77-15049699a626 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:04.094459 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:04.095063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:04.095339 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:04.095598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:04.105465 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:04.106877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:04.116244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1439/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:04.733266 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:04.733907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:04.734043 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:04.766154 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] MFA Rules not processed for user `e5b243e3ae794120ba1a8b3161a17eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:04.767323 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:04.810714 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:04.850937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1434/2873] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:04.855476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:04.868917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d088d-e697-4417-aa33-7d17352c1bf0 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:04.873606 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:04.874278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:04.874626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:04.874949 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:04.875660 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:04.877646 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:04.904534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1440/2874] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:04.908676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:04.920075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c01bcc-6127-4e97-9c84-088858e0c6ea None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:04.977724 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:04.978446 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:04.978589 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:04.978856 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:04.979506 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:04.981416 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:05.002875 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1435/2875] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:04 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:37:05.006995 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:05.016905 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c63d3-24cb-481f-a383-6b0ce1056894 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:05.021604 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:05.022194 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:05.022467 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:05.022734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:05.023357 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:05.025093 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:05.036042 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1441/2876] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:05.040389 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:05.052424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0904ffe-ceac-43d1-8a84-00377adbc384 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:05.059571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:05.060611 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:05.061024 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:05.061400 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] PATH_INFO: `/v3/projects/877426a7724945d3bf563af72399af51/users/99de3ba93c1743249c034b0385c4e583/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:05.080395 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=877426a7724945d3bf563af72399af51, user_id=99de3ba93c1743249c034b0385c4e583, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:05.083903 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:05.101013 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1436/2877] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:05 2022] PUT /identity/v3/projects/877426a7724945d3bf563af72399af51/users/99de3ba93c1743249c034b0385c4e583/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:05.105016 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:05.117381 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec44a2-57a8-4635-95fd-b3fb75868d0e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:05.190265 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:05.190901 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:05.191180 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:05.191453 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:05.192921 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:05.194559 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e1ca0-ce29-4648-896b-976ecb56beab admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:05.207161 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1442/2878] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 07:37:05.212121 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:05.222911 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4a9e-f73a-47f2-9754-d7363a14bdfd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:05.227813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:05.228463 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:05.228732 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:05.229046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] PATH_INFO: `/v3/projects/877426a7724945d3bf563af72399af51/users/99de3ba93c1743249c034b0385c4e583/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:05.230950 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=877426a7724945d3bf563af72399af51, user_id=99de3ba93c1743249c034b0385c4e583, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:05.233601 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:05.253217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1437/2879] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:05 2022] PUT /identity/v3/projects/877426a7724945d3bf563af72399af51/users/99de3ba93c1743249c034b0385c4e583/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:05.873957 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f884d-5106-4239-9051-790176363277 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:05.874564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f884d-5106-4239-9051-790176363277 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:05.874564 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f884d-5106-4239-9051-790176363277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:05.905617 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fd3f884d-5106-4239-9051-790176363277 None None] MFA Rules not processed for user `99de3ba93c1743249c034b0385c4e583`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:05.907124 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd3f884d-5106-4239-9051-790176363277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:05.979307 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1443/2880] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:05.990778 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:05.992713 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:06.052500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf93f-ed8c-456f-af7c-6704dd9b5b6e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:06.056100 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:06.056696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:06.056989 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:06.057256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:06.100113 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:06.115205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:06.116607 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:06.125729 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1438/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:37:06.584752 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f884d-5106-4239-9051-790176363277 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:06.606127 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:06.643916 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f884d-5106-4239-9051-790176363277 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:06.648187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:06.648789 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:06.649101 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:06.649496 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:06.651088 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:06.660976 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:06.662402 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:06.670373 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1444/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 07:37:06.747862 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:06.758092 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37c51-1a0d-4247-b300-0b6f6e0b03bc service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:06.761649 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:06.762242 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:06.762523 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:06.762793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:06.764519 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:06.835579 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:06.837029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:06.848167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1439/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:07.498038 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:07.499405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:07.510495 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9ef6-830c-4379-8760-67cb3440a0fe service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:07.513977 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:07.514598 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:07.514869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:07.515157 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:07.524247 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:07.525779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:07.534897 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1445/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:16.843182 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=42 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:17.530836 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=23 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:19.865190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:19.866586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:19.938883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5446-4e1f-4559-9322-f5e3a390e476 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:19.944890 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:19.945539 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:19.946011 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:19.946653 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:19.948779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:19.961364 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:19.962942 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:19.973468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1440/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:20.178877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:20.195968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d906b-ee53-487a-961e-8598e056dffe service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:20.201027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:20.201628 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:20.201902 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:20.202175 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:20.203918 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:20.217269 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:20.219174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:20.231801 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1446/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:20.764222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:20.835746 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aab1e9-84d2-4788-ab61-0450d4254f76 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:20.841609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:20.842274 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:20.842550 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:20.842849 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] PATH_INFO: `/v3/users/e5b243e3ae794120ba1a8b3161a17eac` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:20.844144 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b243e3ae794120ba1a8b3161a17eac)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:20.846082 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:20.896708 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b243e3ae794120ba1a8b3161a17eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:20.897846 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b243e3ae794120ba1a8b3161a17eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:20.898472 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b243e3ae794120ba1a8b3161a17eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:20.906222 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b243e3ae794120ba1a8b3161a17eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:20.918959 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b243e3ae794120ba1a8b3161a17eac'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:20.931776 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1441/2887] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:20 2022] DELETE /identity/v3/users/e5b243e3ae794120ba1a8b3161a17eac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:21.114123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.125834 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e600398-8c55-4371-ae53-8add992fc6bb service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.186827 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257974-17c0-450f-961a-95011738aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:21.187519 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257974-17c0-450f-961a-95011738aede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.187637 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257974-17c0-450f-961a-95011738aede admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.187876 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257974-17c0-450f-961a-95011738aede admin admin] PATH_INFO: `/v3/projects/88f1e4d66efc498992919fa2540fadb3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.189024 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49257974-17c0-450f-961a-95011738aede admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f1e4d66efc498992919fa2540fadb3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:21.191104 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49257974-17c0-450f-961a-95011738aede admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:21.245224 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f1e4d66efc498992919fa2540fadb3', 'request_id': 'req-49257974-17c0-450f-961a-95011738aede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.250626 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f1e4d66efc498992919fa2540fadb3', 'request_id': 'req-49257974-17c0-450f-961a-95011738aede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.258000 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f1e4d66efc498992919fa2540fadb3', 'request_id': 'req-49257974-17c0-450f-961a-95011738aede'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.269045 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] The token cache is being invalidate because project 88f1e4d66efc498992919fa2540fadb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:21.269355 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.269964 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49257974-17c0-450f-961a-95011738aede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.271431 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1447/2888] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:21 2022] DELETE /identity/v3/projects/88f1e4d66efc498992919fa2540fadb3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:21.275512 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.276902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:21.289929 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2dbb-c2cd-48ed-aa2d-c606bb9940fb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.294468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:21.346790 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:21.347418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.347719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.348051 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] PATH_INFO: `/v3/users/99de3ba93c1743249c034b0385c4e583` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.349217 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99de3ba93c1743249c034b0385c4e583)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:21.351245 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:21.396705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99de3ba93c1743249c034b0385c4e583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.397365 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99de3ba93c1743249c034b0385c4e583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.397960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99de3ba93c1743249c034b0385c4e583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.404793 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99de3ba93c1743249c034b0385c4e583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.416868 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-560e5a00-308b-4955-bb04-b5b07902e936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99de3ba93c1743249c034b0385c4e583'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.429284 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1442/2889] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:21 2022] DELETE /identity/v3/users/99de3ba93c1743249c034b0385c4e583 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:21.597200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257974-17c0-450f-961a-95011738aede admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.598763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49257974-17c0-450f-961a-95011738aede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:21.612401 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257974-17c0-450f-961a-95011738aede admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.673656 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:21.674308 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.674607 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.674877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] PATH_INFO: `/v3/projects/877426a7724945d3bf563af72399af51` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.676066 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=877426a7724945d3bf563af72399af51)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:21.677998 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:21.700842 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:21.726510 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877426a7724945d3bf563af72399af51', 'request_id': 'req-294b459d-0a6c-4607-a09b-d2689bb1fc21'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.732092 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877426a7724945d3bf563af72399af51', 'request_id': 'req-294b459d-0a6c-4607-a09b-d2689bb1fc21'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.739383 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877426a7724945d3bf563af72399af51', 'request_id': 'req-294b459d-0a6c-4607-a09b-d2689bb1fc21'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.750590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] The token cache is being invalidate because project 877426a7724945d3bf563af72399af51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:21.750880 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.751513 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-294b459d-0a6c-4607-a09b-d2689bb1fc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:21.753005 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1448/2890] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:21 2022] DELETE /identity/v3/projects/877426a7724945d3bf563af72399af51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:21.758447 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.758733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.759010 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.780080 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:21.781321 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:21.838773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1443/2891] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:21.846618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.846906 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.847200 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.867352 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:21.868619 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:21.876563 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1449/2892] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:21.881532 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.891286 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344689f3-1b77-4171-8b7c-8dff7e383696 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.896707 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:21.897334 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.897586 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.897851 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.898486 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:37:21.898786 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:21.900377 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:21.911677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1444/2893] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:37:21.916280 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.926130 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb5bd0-a514-4114-b908-caf7e96258b9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.930802 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:21.931393 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:21.931670 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:21.931942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:21.932571 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:21.934776 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:21.962823 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1450/2894] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:21 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:21.966906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:21.977081 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb61d3-616c-492d-a9ec-adf51239bdb3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:21.988715 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:22.033290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.033887 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.034163 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.034439 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.035091 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.036990 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.057428 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1445/2895] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:21 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:22.061821 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.072931 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffe8a-1a3c-4750-b1bb-387ed30841bf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.077917 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.078516 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.078796 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.079087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.079723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.081345 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.097705 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1451/2896] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:22.101748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.111631 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da944c4e-3e6f-4fa1-a8fd-10e023f54ef4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.116895 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.117508 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.117781 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.118046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.118675 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.120229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62983879-7ff2-448a-9244-0bbc23222ed4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.131086 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1446/2897] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:22.135110 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.145187 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498800b-6026-4198-8743-c61b6eb85aa6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.149795 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.150436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.150714 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.150988 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] PATH_INFO: `/v3/projects/5f59d70a5e574fac8c81b2e5320abf7d/users/f8d962037a10476ca7e71e5edfa48de4/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.167009 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f59d70a5e574fac8c81b2e5320abf7d, user_id=f8d962037a10476ca7e71e5edfa48de4, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.169659 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.182779 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1452/2898] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:22 2022] PUT /identity/v3/projects/5f59d70a5e574fac8c81b2e5320abf7d/users/f8d962037a10476ca7e71e5edfa48de4/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:22.195514 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.195840 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.196111 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.224062 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] MFA Rules not processed for user `f8d962037a10476ca7e71e5edfa48de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:22.225285 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:22.286777 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1447/2899] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:22.291244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.352731 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b5560-92b9-4d03-8bef-bd42e9e89ac4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.357334 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.357927 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.358198 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.358462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.359081 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.360968 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.386965 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1453/2900] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:22 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:22.391093 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.401556 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20896be-6481-4a55-ae3b-460368e511fc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.456968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.457605 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.457952 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.458221 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.462214 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.465001 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.486122 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1448/2901] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:22 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:22.491159 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.501337 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f195b4-615a-4b9f-b379-05e74c195719 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.506267 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.506859 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.507138 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.507410 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.508055 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.509690 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.520673 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1454/2902] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:22.524630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.534387 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684857ee-5667-4022-b28f-f403bf63f1a1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.539937 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.540494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.540765 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.541075 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] PATH_INFO: `/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe/users/76790305c90748c7814d369de3023ac6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.552382 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:22.557077 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b90c5575b7d44e39fca7c3ac263fcbe, user_id=76790305c90748c7814d369de3023ac6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.560737 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.574630 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1449/2903] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:22 2022] PUT /identity/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe/users/76790305c90748c7814d369de3023ac6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:37:22.578688 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.589353 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e514c-c9fa-4d72-9afd-17a9a57200e6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.601939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:22.646323 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.646926 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.647203 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.647473 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.648109 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.649734 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4d045-2791-4c74-be5a-3a977d313e86 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.659314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1455/2904] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:22.664330 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.674639 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7170f9-da98-4e98-99e3-1f5815847e63 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.679343 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.679979 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.680256 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.680527 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] PATH_INFO: `/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe/users/76790305c90748c7814d369de3023ac6/roles/30776d1972a74ad8b38be970aa4d6478` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.682461 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b90c5575b7d44e39fca7c3ac263fcbe, user_id=76790305c90748c7814d369de3023ac6, role_id=30776d1972a74ad8b38be970aa4d6478)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.685350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.695748 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1450/2905] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:22 2022] PUT /identity/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe/users/76790305c90748c7814d369de3023ac6/roles/30776d1972a74ad8b38be970aa4d6478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:37:22.706047 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.706405 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.707215 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.734932 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] MFA Rules not processed for user `76790305c90748c7814d369de3023ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:22.736148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:22.804515 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1456/2906] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:22.815197 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:22.817130 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:22.874373 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdf27e-dc17-4202-92fa-057ea0e90460 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:22.878022 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:22.879326 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:22.879803 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:22.880084 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:22.937329 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:22.938789 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:22.948554 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1451/2907] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:27.165751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:27.167223 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:27.179129 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52192b17-a255-4524-b878-9c7741b65ee2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:27.182599 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:27.183193 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:27.183468 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:27.183733 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:27.193061 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:27.194462 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3271708-6776-47c3-a87f-819c8697f32a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:27.203618 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1457/2908] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:32.096476 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:32.163118 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b67d0-634f-4ca8-a603-8585d7043c8f service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:32.167534 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:32.168148 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:32.168424 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:32.168695 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] PATH_INFO: `/v3/users/f8d962037a10476ca7e71e5edfa48de4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:32.169835 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d962037a10476ca7e71e5edfa48de4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:32.171709 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:32.205996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:32.225260 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d962037a10476ca7e71e5edfa48de4'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.225910 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d962037a10476ca7e71e5edfa48de4'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.226511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d962037a10476ca7e71e5edfa48de4'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.233270 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d962037a10476ca7e71e5edfa48de4'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.245185 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d962037a10476ca7e71e5edfa48de4'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.259779 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1452/2909] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:32 2022] DELETE /identity/v3/users/f8d962037a10476ca7e71e5edfa48de4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:32.264561 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:32.265089 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:32.265205 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:32.284107 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:32.285338 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:32.343912 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1458/2910] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:32.353546 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:32.354906 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:32.411136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293e476-c75e-4fd1-b577-fe9beafc6439 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:32.414896 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:32.415432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:32.415706 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:32.415980 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:32.424686 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:32.426132 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:32.436164 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1453/2911] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:32.705384 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:32.706794 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:32.720068 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479acdba-c739-4aa8-b54b-8394b1f008fb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:32.725356 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:32.725986 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:32.726262 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:32.726534 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] PATH_INFO: `/v3/projects/5f59d70a5e574fac8c81b2e5320abf7d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:32.727745 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f59d70a5e574fac8c81b2e5320abf7d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:32.729763 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:32.753244 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:32.777558 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f59d70a5e574fac8c81b2e5320abf7d', 'request_id': 'req-5e45dd52-3a38-4e78-91c2-f92028bec29b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.783137 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f59d70a5e574fac8c81b2e5320abf7d', 'request_id': 'req-5e45dd52-3a38-4e78-91c2-f92028bec29b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.790866 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f59d70a5e574fac8c81b2e5320abf7d', 'request_id': 'req-5e45dd52-3a38-4e78-91c2-f92028bec29b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.801891 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] The token cache is being invalidate because project 5f59d70a5e574fac8c81b2e5320abf7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:32.802177 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.803523 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.805271 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1459/2912] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:32 2022] DELETE /identity/v3/projects/5f59d70a5e574fac8c81b2e5320abf7d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:32.809071 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:32.819342 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532840-d9b1-48d9-8c81-c866470947d6 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:32.874787 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:32.875401 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:32.875677 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:32.875947 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] PATH_INFO: `/v3/users/76790305c90748c7814d369de3023ac6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:32.877088 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76790305c90748c7814d369de3023ac6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:32.878973 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:32.923199 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76790305c90748c7814d369de3023ac6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.923844 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76790305c90748c7814d369de3023ac6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.924432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76790305c90748c7814d369de3023ac6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.930318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:32.932664 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76790305c90748c7814d369de3023ac6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.951661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cfd67092-d5ac-44a9-8a29-4d799391eff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76790305c90748c7814d369de3023ac6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:32.966920 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1454/2913] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:32 2022] DELETE /identity/v3/users/76790305c90748c7814d369de3023ac6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:33.234152 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.235581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:33.248787 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45dd52-3a38-4e78-91c2-f92028bec29b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.307751 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.308478 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.308642 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.308910 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] PATH_INFO: `/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.310294 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b90c5575b7d44e39fca7c3ac263fcbe)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.312183 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.357735 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b90c5575b7d44e39fca7c3ac263fcbe', 'request_id': 'req-54485e8f-67da-4a5d-8195-537ea63ce3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:33.363239 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b90c5575b7d44e39fca7c3ac263fcbe', 'request_id': 'req-54485e8f-67da-4a5d-8195-537ea63ce3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:33.369877 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b90c5575b7d44e39fca7c3ac263fcbe', 'request_id': 'req-54485e8f-67da-4a5d-8195-537ea63ce3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:33.381686 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] The token cache is being invalidate because project 3b90c5575b7d44e39fca7c3ac263fcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:33.381971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:33.382592 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-54485e8f-67da-4a5d-8195-537ea63ce3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:33.384048 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1460/2914] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:33 2022] DELETE /identity/v3/projects/3b90c5575b7d44e39fca7c3ac263fcbe => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:33.389902 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.390445 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.390567 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.410500 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:33.411733 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:33.470609 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1455/2915] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:33.479116 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.479333 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.479600 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.499095 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:33.500293 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:33.508869 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1461/2916] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:33.512967 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.523058 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3736e0-6459-4e32-a203-11642e2f7b0c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.527576 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.528159 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.528432 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.528704 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.529367 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 07:37:33.529668 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.531229 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.542494 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1456/2917] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 07:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 07:37:33.546757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.556027 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd92d-5a98-48c6-bb7b-22a3099bf518 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.560820 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.561433 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.561706 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.561971 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.562583 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.564449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.592817 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1462/2918] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 07:37:33 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 07:37:33.597136 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.607290 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3c13f-9b51-4a66-98a5-ea2273d526e6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.663311 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.663913 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.664188 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.664457 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.665126 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.667016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.688085 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1457/2919] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 07:37:33 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 07:37:33.692380 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.703665 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9bcb3-9267-49f0-b69d-8ee1dd81458f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.708276 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.709010 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.709314 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.709638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.710278 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.711865 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.717131 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:33.723236 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1463/2920] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:33.727753 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.737759 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b33f-ba00-4e70-929d-6539eb69d047 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.742888 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.743475 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.743749 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.744016 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.744676 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.746288 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f556-5314-489e-807d-c61296dac7db admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.757491 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1458/2921] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 07:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 07:37:33.761588 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.771497 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a289-aeb3-4863-8275-726267ab6f5e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.776331 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.777001 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.777282 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.777553 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] PATH_INFO: `/v3/projects/960974705d6c4674b339d9806ac5decb/users/25d6c58513c24d6d8c5ecf9c356d5342/roles/e036f4b567c14abab4c3011e4611ab47` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.792063 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=960974705d6c4674b339d9806ac5decb, user_id=25d6c58513c24d6d8c5ecf9c356d5342, role_id=e036f4b567c14abab4c3011e4611ab47)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.794702 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:33.807360 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1464/2922] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 07:37:33 2022] PUT /identity/v3/projects/960974705d6c4674b339d9806ac5decb/users/25d6c58513c24d6d8c5ecf9c356d5342/roles/e036f4b567c14abab4c3011e4611ab47 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:33.820719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.821037 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.821337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.848350 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] MFA Rules not processed for user `25d6c58513c24d6d8c5ecf9c356d5342`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:33.849610 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:33.875337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:33.915241 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1459/2923] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 07:37:33.926651 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:33.929147 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:33.984548 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32a0e-6780-4972-ab5f-222402eaec5c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:33.988303 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce964ecd12bd4e3d9d85bf5dc227d35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:33.988841 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:33.989148 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:33.989418 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:33.998123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:33.999533 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ba922-45c2-450d-859f-ef0a1c357695 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:34.008757 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1465/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:34.033492 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:34.094048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24c7c8-81ff-422c-8552-3ee68cc5803f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:34.098561 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:34.099168 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:34.100096 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:34.100385 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] PATH_INFO: `/v3/users/25d6c58513c24d6d8c5ecf9c356d5342` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:34.101520 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d6c58513c24d6d8c5ecf9c356d5342)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:34.103398 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:34.147996 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d6c58513c24d6d8c5ecf9c356d5342'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.148655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d6c58513c24d6d8c5ecf9c356d5342'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.149312 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d6c58513c24d6d8c5ecf9c356d5342'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.155978 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d6c58513c24d6d8c5ecf9c356d5342'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.168511 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d6c58513c24d6d8c5ecf9c356d5342'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.182018 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1460/2925] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:37:34 2022] DELETE /identity/v3/users/25d6c58513c24d6d8c5ecf9c356d5342 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:34.186627 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:34.186907 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:34.187174 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:34.206246 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] MFA Rules not processed for user `955dc67368494986a8ad78d63a857238`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 07:37:34.207426 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:34.265774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1466/2926] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 07:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 07:37:34.275667 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:34.277103 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:34.329190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:34.336048 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c0fe-2557-476f-9dc7-6f4b6dfcd575 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:34.339894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:34.340483 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:34.340750 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:34.341038 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:34.350262 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:34.351655 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:34.360805 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1461/2927] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:37:34.626845 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:37:34.628329 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:37:34.641250 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fca473-c2dd-45c1-89b0-7dbc2a754947 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:37:34.645806 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:37:34.647097 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:37:34.647375 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:37:34.647639 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] PATH_INFO: `/v3/projects/960974705d6c4674b339d9806ac5decb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:37:34.648753 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960974705d6c4674b339d9806ac5decb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:37:34.650730 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:37:34.697362 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960974705d6c4674b339d9806ac5decb', 'request_id': 'req-8096b690-6ae8-419b-86de-4fa99b3e5d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.703071 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960974705d6c4674b339d9806ac5decb', 'request_id': 'req-8096b690-6ae8-419b-86de-4fa99b3e5d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.711328 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960974705d6c4674b339d9806ac5decb', 'request_id': 'req-8096b690-6ae8-419b-86de-4fa99b3e5d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.722632 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] The token cache is being invalidate because project 960974705d6c4674b339d9806ac5decb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:37:34.722915 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.723638 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:37:34.725738 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1467/2928] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:37:34 2022] DELETE /identity/v3/projects/960974705d6c4674b339d9806ac5decb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:37:44.356719 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:37:44.719647 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:13.086423 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:13.087883 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:13.099819 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb78f7-26d2-405b-8366-fbe9a50baae9 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:13.148106 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:13.148698 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:13.149004 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:13.149276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:13.151026 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:13.212468 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:13.213832 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:13.223189 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1462/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:38:19.190774 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:19.193291 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:19.208538 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096b690-6ae8-419b-86de-4fa99b3e5d05 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:19.212581 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:19.213257 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:19.213537 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:19.213808 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] PATH_INFO: `/v3/users/04cf365443b44d9d87338d95b5b0251b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:19.215298 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04cf365443b44d9d87338d95b5b0251b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:19.216894 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:19.268427 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04cf365443b44d9d87338d95b5b0251b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.269098 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04cf365443b44d9d87338d95b5b0251b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.277371 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04cf365443b44d9d87338d95b5b0251b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.278073 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04cf365443b44d9d87338d95b5b0251b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.290442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04cf365443b44d9d87338d95b5b0251b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.305713 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1468/2930] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:38:19 2022] DELETE /identity/v3/users/04cf365443b44d9d87338d95b5b0251b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:19.481050 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:19.491865 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca64a6-0c99-430f-8339-2f5094fd8531 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:19.547894 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:19.548517 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:19.548792 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:19.549099 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] PATH_INFO: `/v3/projects/bfa33399f5994fb4a1ad804fa6e75f8b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:19.550205 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa33399f5994fb4a1ad804fa6e75f8b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:19.552788 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:19.596339 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa33399f5994fb4a1ad804fa6e75f8b', 'request_id': 'req-443d6c4b-88b5-4b50-94c3-a816afecb905'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.598254 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:19.603442 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa33399f5994fb4a1ad804fa6e75f8b', 'request_id': 'req-443d6c4b-88b5-4b50-94c3-a816afecb905'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.611987 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa33399f5994fb4a1ad804fa6e75f8b', 'request_id': 'req-443d6c4b-88b5-4b50-94c3-a816afecb905'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.623907 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] The token cache is being invalidate because project bfa33399f5994fb4a1ad804fa6e75f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:38:19.624190 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.624800 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.626276 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1463/2931] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:38:19 2022] DELETE /identity/v3/projects/bfa33399f5994fb4a1ad804fa6e75f8b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:19.630316 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:19.631723 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:19.644936 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b6c2e-9f56-49b0-8df8-21bc291355c4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:19.701041 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:19.701780 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:19.701930 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:19.702197 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] PATH_INFO: `/v3/users/0a672af929f3442e88d31d2d8dc06ed9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:19.703461 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a672af929f3442e88d31d2d8dc06ed9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:19.705634 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:19.751590 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a672af929f3442e88d31d2d8dc06ed9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.752151 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a672af929f3442e88d31d2d8dc06ed9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.759698 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a672af929f3442e88d31d2d8dc06ed9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.760346 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a672af929f3442e88d31d2d8dc06ed9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.772768 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a672af929f3442e88d31d2d8dc06ed9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:19.774476 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:19.787505 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1469/2932] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:38:19 2022] DELETE /identity/v3/users/0a672af929f3442e88d31d2d8dc06ed9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:19.959194 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:19.960661 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:19.974337 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6c4b-88b5-4b50-94c3-a816afecb905 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:20.032146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:20.032771 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:20.033078 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:20.033348 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] PATH_INFO: `/v3/projects/861b07d115dd445786956f32e8c94261` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:20.034440 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861b07d115dd445786956f32e8c94261)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:20.036488 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:20.081772 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861b07d115dd445786956f32e8c94261', 'request_id': 'req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:20.087272 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861b07d115dd445786956f32e8c94261', 'request_id': 'req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:20.094503 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861b07d115dd445786956f32e8c94261', 'request_id': 'req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:20.105355 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] The token cache is being invalidate because project 861b07d115dd445786956f32e8c94261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:38:20.105642 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:20.106269 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:20.107716 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1464/2933] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:38:19 2022] DELETE /identity/v3/projects/861b07d115dd445786956f32e8c94261 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:29.784374 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:30.102318 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:43.163911 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:43.165406 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:43.220710 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706d32b-1597-479c-93a8-40b3b5ddfe50 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:43.266937 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '170658d3fbf547a09a4dd2bb77ecb01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:43.267527 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:43.267800 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:43.268069 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:43.269873 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:43.332123 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:43.334277 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:43.343508 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1470/2934] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:38:49.482198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:49.483813 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:49.500335 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4654-9904-4516-80c3-31e4cb0a7ac8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:49.505650 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:49.506346 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:49.506626 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:49.506925 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] PATH_INFO: `/v3/users/880d5318ebb145fbb3cd8b02814903f2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:49.508049 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=880d5318ebb145fbb3cd8b02814903f2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:49.510107 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:49.555420 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880d5318ebb145fbb3cd8b02814903f2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.556076 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880d5318ebb145fbb3cd8b02814903f2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.556696 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880d5318ebb145fbb3cd8b02814903f2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.563797 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880d5318ebb145fbb3cd8b02814903f2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.576304 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880d5318ebb145fbb3cd8b02814903f2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.590146 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1465/2935] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:38:49 2022] DELETE /identity/v3/users/880d5318ebb145fbb3cd8b02814903f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 07:38:49.761707 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:49.773570 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece96d31-7f86-42fd-8cdc-8ff27ea4f3ec service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:49.833315 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:49.834025 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:49.834300 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:49.834569 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] PATH_INFO: `/v3/projects/39ec4a1f0d60440c86413595a8eb16ae` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:49.835685 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ec4a1f0d60440c86413595a8eb16ae)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:49.837630 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:49.845449 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:49.883699 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ec4a1f0d60440c86413595a8eb16ae', 'request_id': 'req-59729577-c5a2-4d12-a95e-7c63d4f840ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.889166 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ec4a1f0d60440c86413595a8eb16ae', 'request_id': 'req-59729577-c5a2-4d12-a95e-7c63d4f840ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.896721 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ec4a1f0d60440c86413595a8eb16ae', 'request_id': 'req-59729577-c5a2-4d12-a95e-7c63d4f840ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.907550 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] The token cache is being invalidate because project 39ec4a1f0d60440c86413595a8eb16ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:38:49.907829 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.908442 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:49.909908 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1471/2936] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:38:49 2022] DELETE /identity/v3/projects/39ec4a1f0d60440c86413595a8eb16ae => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:49.914507 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:49.915968 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:49.929108 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ca1a-c012-435d-910c-94f749879e6a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:49.984404 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:49.985029 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:49.985306 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:49.985575 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] PATH_INFO: `/v3/users/b5d4df6ebec743b991eac4eee5cad9bf` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:49.986687 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d4df6ebec743b991eac4eee5cad9bf)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:49.988568 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:50.034773 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d4df6ebec743b991eac4eee5cad9bf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.035418 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d4df6ebec743b991eac4eee5cad9bf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.036064 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d4df6ebec743b991eac4eee5cad9bf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.042822 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d4df6ebec743b991eac4eee5cad9bf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.054838 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d4df6ebec743b991eac4eee5cad9bf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.056426 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:38:50.074470 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1466/2937] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 07:38:49 2022] DELETE /identity/v3/users/b5d4df6ebec743b991eac4eee5cad9bf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:38:50.248436 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:38:50.249928 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:38:50.263974 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59729577-c5a2-4d12-a95e-7c63d4f840ff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:38:50.322852 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '955dc67368494986a8ad78d63a857238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '712d7cadf2c0478da279cd0f1a4dd259', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:38:50.323399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:38:50.323675 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:38:50.323942 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] PATH_INFO: `/v3/projects/365db8c3711b4a718745dfd83a505234` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:38:50.325087 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] RBAC: Authorizing `identity:delete_project(project_id=365db8c3711b4a718745dfd83a505234)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:38:50.326975 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:38:50.372613 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365db8c3711b4a718745dfd83a505234', 'request_id': 'req-0d4dc641-2796-4dcf-a0af-54f0cf20428c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.378082 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365db8c3711b4a718745dfd83a505234', 'request_id': 'req-0d4dc641-2796-4dcf-a0af-54f0cf20428c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.385417 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365db8c3711b4a718745dfd83a505234', 'request_id': 'req-0d4dc641-2796-4dcf-a0af-54f0cf20428c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.397019 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] The token cache is being invalidate because project 365db8c3711b4a718745dfd83a505234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 07:38:50.397313 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.397939 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0d4dc641-2796-4dcf-a0af-54f0cf20428c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 07:38:50.399399 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1472/2938] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 07:38:50 2022] DELETE /identity/v3/projects/365db8c3711b4a718745dfd83a505234 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 07:39:00.066183 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:39:00.393674 dv-cinder-859447-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 07:42:21.592705 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 07:42:21.594135 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:42:21.738823 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7f57-ae55-4ec9-8260-bd8935ed5fed admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 07:42:21.798046 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cbc4b10d4745d08bb05dbd7a36dc84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8c197c9445d453dbdf730b9d157cf14', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 07:42:21.798681 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 07:42:21.798960 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 07:42:21.799232 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 07:42:21.801224 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 07:42:21.814597 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 07:42:21.816167 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5fa21-6f1d-4f1e-bcb1-8b2d6838381e service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 07:42:21.827198 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1467/2939] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 07:42:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 07:42:31.822571 dv-cinder-859447-1-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}